1 Reply Latest reply: May 21, 2009 9:24 AM by 843829 RSS

    shared memory / native stack question

    843829
      Hope this is the right forum...

      Background:
      Environment – Java 1.5.0_11-b03 (32 bit vm), Solaris 10, M5000 server (64 cpu/128GB)
      Application – custom webserver (more or less), uses java threads, uses shared memory through JNI interface.
      Memory sizing: –Xms512M –Xmx1024M, usual running heap max at ~300M , ~164 threads

      After running for a few days we are encountering OutOfMemoryErrors when attempting to allocate new threads:
      java.lang.OutOfMemoryError: unable to create new native thread
      at java.lang.Thread.start0(Native Method)
      at java.lang.Thread.start(Unknown Source)
      …

      It is my understanding that this is a stack space issue as there is plenty of available heap.
      We are allocating shared memory (fixed size (16k) sliding windows over ~400 files)
      After the thread start failures we then start seeing shared memory allocation issues
      OS_ERR = (12): Not enough space
      (there is plenty of space on the file system, and I expect within the ~4GB process space)

      We can make the issue occur quicker by allocating more shared memory. (still well under 4GB)
      The current environment should be < 1GB shared memory.
      The overall starting memory consumption is < 2GB

      We are planning on reducing the default stack size but have yet to run a test with that since these errors are currently only occurring in production environments.

      Questions:
      Could this be a fragmentation issue?
      Is there any documentation that would help me understand where shared mem gets allocated and where native stacks live? (since they seem to be competing for space)
      Any other hints or tools we could use to help diagnose this problem?

      Thanks!