>> combination of SGA + pga ?
Yes. Total memory footprint for the instance can not exceed 1GiB, sga + pga is pretty much the definition of the instance memory size.
>> include temp, undo, system, sysaux ?
Those tablespaces should not count towards the user database 11G limit. System and sysaux are for the instance catalog and resource monitoring/tuning utilities, don't put stuff there. Just don't. Using system or sys schemas for storing user data is just plain bad form.