Hi all,
We're running Oracle 11.2.0.4 (and 11.2.0.3) in a zone on SPARC Solaris 11.1. Our DBA has recently created a new database instance which won't start. When she attempts to start it in nomount from a pfile she receives the following error:
SQL> startup nomount pfile='/u01/app/oracle/product/rdbms/11.2.0.4/dbs/initMOSMIG.ora';
ORA-03113: end-of-file on communication channel
The alertlog contains the following:
Errors in file /u01/app/oracle/diag/rdbms/mosmig/MOSMIG/trace/MOSMIG_gen0_6281.trc (incident=1249):
ORA-00600: internal error code, arguments: [kslwt_validate_gwaitctx: magic], [0], [], [], [], [], [], [], [], [], [], []
Tue May 06 15:25:02 2014
Sweep [inc][553]: completed
Sweep [inc][417]: completed
Sweep [inc2][553]: completed
Sweep [inc2][417]: completed
Errors in file /u01/app/oracle/diag/rdbms/mosmig/MOSMIG/trace/MOSMIG_mmon_6294.trc (incident=1337):
ORA-00600: internal error code, arguments: [kslgetl:1], [0x4601CB808], [24668], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/mosmig/MOSMIG/incident/incdir_1337/MOSMIG_mmon_6294_i1337.trc
Errors in file /u01/app/oracle/diag/rdbms/mosmig/MOSMIG/trace/MOSMIG_diag_6282.trc (incident=1257):
ORA-00600: internal error code, arguments: [kslwt_validate_gwaitctx: magic], [0], [], [], [], [], [], [], [], [], [], []
Errors in file /u01/app/oracle/diag/rdbms/mosmig/MOSMIG/trace/MOSMIG_dia0_6284.trc (incident=1273):
ORA-00600: internal error code, arguments: [kslwt_validate_gwaitctx: magic], [0], [], [], [], [], [], [], [], [], [], []
Exception [type: SIGBUS, Invalid address alignment] [ADDR:0x6529005F73696EA5] [PC:0x1034DE0B4, kewa_sample_session()+884] [flags: 0x0, count: 1]
Errors in file /u01/app/oracle/diag/rdbms/mosmig/MOSMIG/trace/MOSMIG_mmnl_6295.trc (incident=1345):
ORA-07445: exception encountered: core dump [kewa_sample_session()+884] [SIGBUS] [ADDR:0x6529005F73696EA5] [PC:0x1034DE0B4] [Invalid address alignment] []
Incident details in: /u01/app/oracle/diag/rdbms/mosmig/MOSMIG/incident/incdir_1345/MOSMIG_mmnl_6295_i1345.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Exception [type: SIGBUS, Invalid address alignment] [ADDR:0x5F6C6D5F6C6F7AD3] [PC:0x105747730, kkslof()+16] [flags: 0x0, count: 1]
Errors in file /u01/app/oracle/diag/rdbms/mosmig/MOSMIG/trace/MOSMIG_pmon_6269.trc (incident=1225):
ORA-07445: exception encountered: core dump [kkslof()+16] [SIGBUS] [ADDR:0x5F6C6D5F6C6F7AD3] [PC:0x105747730] [Invalid address alignment] []
Incident details in: /u01/app/oracle/diag/rdbms/mosmig/MOSMIG/incident/incdir_1225/MOSMIG_pmon_6269_i1225.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Exception [type: SIGSEGV, Address not mapped to object] [ADDR:0x1A3452AA20] [PC:0x100AAE9B4, kslwh_dmp_sess_waithist()+1492] [flags: 0x0, count: 1]
Errors in file /u01/app/oracle/diag/rdbms/mosmig/MOSMIG/trace/MOSMIG_mmon_6294.trc (incident=1338):
ORA-07445: exception encountered: core dump [kslwh_dmp_sess_waithist()+1492] [SIGSEGV] [ADDR:0x1A3452AA20] [PC:0x100AAE9B4] [Address not mapped to object] []
ORA-00600: internal error code, arguments: [kslgetl:1], [0x4601CB808], [24668], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/mosmig/MOSMIG/incident/incdir_1338/MOSMIG_mmon_6294_i1338.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Exception [type: SIGSEGV, Address not mapped to object] [ADDR:0x17582099C0] [PC:0x100AAE9B4, kslwh_dmp_sess_waithist()+1492] [flags: 0x0, count: 2]
Errors in file /u01/app/oracle/diag/rdbms/mosmig/MOSMIG/trace/MOSMIG_mmnl_6295.trc (incident=1346):
ORA-07445: exception encountered: core dump [kslwh_dmp_sess_waithist()+1492] [SIGSEGV] [ADDR:0x17582099C0] [PC:0x100AAE9B4] [Address not mapped to object] []
ORA-07445: exception encountered: core dump [kewa_sample_session()+884] [SIGBUS] [ADDR:0x6529005F73696EA5] [PC:0x1034DE0B4] [Invalid address alignment] []
Incident details in: /u01/app/oracle/diag/rdbms/mosmig/MOSMIG/incident/incdir_1346/MOSMIG_mmnl_6295_i1346.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Exception [type: SIGSEGV, Address not mapped to object] [ADDR:0x9090000812400001] [PC:0xFFFFFFFF1C05C034, strlen()+20] [flags: 0x0, count: 3]
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Errors in file /u01/app/oracle/diag/rdbms/mosmig/MOSMIG/trace/MOSMIG_smon_6292.trc (incident=1321):
ORA-00600: internal error code, arguments: [kslwt_validate_gwaitctx: magic], [0], [], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/mosmig/MOSMIG/incident/incdir_1321/MOSMIG_smon_6292_i1321.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
Errors in file /u01/app/oracle/diag/rdbms/mosmig/MOSMIG/trace/MOSMIG_dbw0_6286.trc (incident=1289):
ORA-00600: internal error code, arguments: [kslwt_validate_gwaitctx: magic], [0], [], [], [], [], [], [], [], [], [], []
There are several exception messages in there making reference to addressing, and because this problem occurs during the parameter setting stage, the DBA is of the opinion that these are memory problems However, the physical server has 255GB of memory and we are not pushing it anywhere near to its capacity. I've checked the resource controls for the zone server and even increased the values to allocate additional memory as follows, but all to no avail:
# prctl $$
process: 17198: -bash
NAME PRIVILEGE VALUE FLAG ACTION RECIPIENT
process.max-sigqueue-size
basic 128 - deny 17198
privileged 512 - deny -
system 8.19K max deny -
process.max-itimers
privileged 1.00K - deny -
system 65.5K max deny -
process.max-port-events
privileged 65.5K - deny -
system 2.15G max deny -
process.max-msg-messages
privileged 8.19K - deny -
system 4.29G max deny -
process.max-msg-qbytes
privileged 64.0KB - deny -
system 16.0EB max deny -
process.max-sem-ops
privileged 512 - deny -
system 2.15G max deny -
process.max-sem-nsems
privileged 512 - deny -
system 32.8K max deny -
process.max-address-space
privileged 16.0EB max deny -
system 16.0EB max deny -
process.max-file-descriptor
basic 256 - deny 17198
privileged 65.5K - deny -
system 2.15G max deny -
process.max-core-size
privileged 8.00EB max deny -
system 8.00EB max deny -
process.max-stack-size
basic 8.00MB - deny 17198
privileged 8.00EB max deny -
system 8.00EB max deny -
process.max-data-size
privileged 16.0EB max deny -
system 16.0EB max deny -
process.max-file-size
privileged 8.00EB max deny,signal=XFSZ -
system 8.00EB max deny -
process.max-cpu-time
privileged 18.4Es inf signal=XCPU -
system 18.4Es inf none -
task.max-cpu-time
usage 0s
system 18.4Es inf none -
task.max-processes
usage 3
system 2.15G max deny -
task.max-lwps
usage 4
system 2.15G max deny -
project.max-contracts
privileged 10.0K - deny -
system 2.15G max deny -
project.max-locked-memory
usage 12.1GB
system 16.0EB max deny -
project.max-port-ids
privileged 8.19K - deny -
system 65.5K max deny -
project.max-shm-memory
privileged 150GB - deny -
system 16.0EB max deny -
project.max-shm-ids
privileged 128 - deny -
system 16.8M max deny -
project.max-msg-ids
privileged 128 - deny -
system 16.8M max deny -
project.max-sem-ids
privileged 128 - deny -
system 16.8M max deny -
project.max-crypto-memory
usage 32.8KB
privileged 62.7GB - deny -
system 16.0EB max deny -
project.max-tasks
usage 6
system 2.15G max deny -
project.max-processes
usage 295
system 2.15G max deny -
project.max-lwps
usage 2.55K
system 2.15G max deny -
project.cpu-cap
usage 0
system 4.29G inf deny -
project.cpu-shares
usage 1
privileged 1 - none -
system 65.5K max none -
zone.max-lofi
usage 0
system 18.4E max deny -
zone.max-swap
usage 21.9GB
system 16.0EB max deny -
zone.max-locked-memory
usage 12.1GB
system 16.0EB max deny -
zone.max-shm-memory
usage 18.1GB
privileged 150GB - deny -
system 16.0EB max deny -
zone.max-shm-ids
usage 21
system 16.8M max deny -
zone.max-sem-ids
usage 7
system 16.8M max deny -
zone.max-msg-ids
usage 0
system 16.8M max deny -
zone.max-processes
usage 336
system 2.15G max deny -
zone.max-lwps
usage 2.70K
system 2.15G max deny -
zone.cpu-cap
usage 0
system 4.29G inf deny -
zone.cpu-shares
usage 1
privileged 1 - none -
system 65.5K max none -
We have several other databases running on this box, but this one won't start. Can anyone suggest what may be wrong here?
Thank you.