ORA-27102 SVR4 Error: 12: Not enough space

Good thoughts, but we had ruled that out early: Filesystem's not the issue, the disk had plenty of space. Metalink itself (472813.1) points to "the Unix error number (ERRNO) 12 during a Unix write()/open() system call, and this Unix error indicates a lack of *process memory* rather than physical disk space."

- /var/adm/messages has no memory- or disk-related messages around the time of failure.
- SAN administrator saw nothing in their logs at the time of failure

We had already tried raising SGA and raising shared memory in the solaris project, but it seems like we're fishing for an answer by blindly raising a parameter when we don't know what OS limit Oracle had reached. The key numbers I'm looking for are those specified in the 'additional information' section. Oracle's knowledge base has nothing that I can use so far.

Thanks! We'd definitely been checking google for information for the past couple weeks before checking with Oracle Forums. In fact, there are several blogs and [lazydba/experts-exchange links|http://www.lazydba.com/oracle/0__125336.html] on the subject which point us in the right direction and was the basis for us looking at enlarging the shared memory kernel parameters to start.

At this point, it's more how to interpret how Oracle spits out information, since there wasn't any publicly available information on the format of the error code.

Much like how P1, P2, and P3 in v$session_wait will mean different things, I would guess that the "Additional Information" tokens after the "Error: 12 code" mean different things. That much is evident in my searches, where it appears that:
Error code 12 => memory related things
Error code 11 => resource is temporarily unavailable for whatever reason
Error code 5 => disk/IO issue

.. so drilling down further, Error code 12's two additional information items must mean something:
-1 => some return code?
8192 => the number at which it failed at? Some bit-wise address?

At no point does the stack gets mentioned in our diagnostic text, which is why I'm asking the larger oracle community.

To elaborate, the filesystem is UFS-based, all configured in RAID-6 [(striped disks with dual parity)|http://en.wikipedia.org/wiki/Redundant_array_of_independent_disks] (including the redo logs and archive logs). The storage underneath the filesystem is a Hitachi 9985v SAN, meaning the physical disks themselves were grouped into logical partitions and then divvied up into filesystems. Files #6 & 57 are on different filesystems, but all of them had ample space at the time.

I am entirely with you on this.
-- if it was exclusively an oracle memory error, we would have seen an ORA-4031 indicating exactly which pool was compromised.
-- if it was a dbwr error, we would have seen a more descriptive alert saying 'disk full' or 'crc did not match'
-- etc.

The only IO activity we did catch was the fact that RMAN archive log backups were running at the time. Our I/O usage charts for those filesystems (and solaris sar/vmstat/iostat counts) did not spike during those times.

Regarding the filesystem setup, our vote was to use ZFS, but this was a decision made beyond our heads. Unfortunately, without a real error to show the SAN administrator, we are unable to provide them effective evidence to support the claim. Getting proper diagnostic information was the point of this forum post -- notably, Metalink's own article +(22080.1 - An Introduction to Error Message Articles)+ pretty much admits we need to look further than just the error codes:

Please note that the steps included under these additional headings
is generally very terse. Information may be incomplete and may use
abbreviations which are not self explanatory. It is also possible that
there may be references to articles which are not visible to customers.
These additional notes are intended to help give pointers and are NOT
intended to be a complete explanation of the error.

More complete descriptions may be included in separate problem/solution or
bulletin documents.