This discussion is archived
6 Replies Latest reply: Mar 14, 2012 11:50 PM by 915071 RSS

Exception in thread "RMI

915071 Newbie
Currently Being Moderated
Hello, I configure dbcache 90m for my data and heap size is 150m, but when I run multi-thread update on my store, the exception happened, and I can't restart the SN.
by the way, I have followed the introduction for compute dbcache and heap size.


03-09-12 11:33:19:81 UTC+8 INFO [sn1] rg1-rn1: ProcessMonitor: Exception in thread "RMI TCP Connection(idle)" Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
03-09-12 11:33:19:83 UTC+8 INFO [sn1] rg1-rn1: ProcessMonitor: Process exiting due to fault
03-09-12 11:33:19:83 UTC+8 INFO [sn1] rg1-rn1: ProcessMonitor: Process exiting due to fault
03-09-12 11:33:19:83 UTC+8 INFO [sn1] rg1-rn1: ProcessMonitor: java.lang.OutOfMemoryError: Java heap space
03-09-12 11:33:19:83 UTC+8 INFO [sn1] rg1-rn1: ProcessMonitor: java.lang.OutOfMemoryError: Java heap space
03-09-12 11:33:20:95 UTC+8 INFO [sn1] rg1-rn1: ProcessMonitor: IOThread exiting
03-09-12 11:33:20:95 UTC+8 INFO [sn1] rg1-rn1: ProcessMonitor: exited, exit code: 200
03-09-12 11:33:23:45 UTC+8 INFO [admin1] [sn1] rg1-rn1: Service status: ERROR_RESTARTING 03-09-12 11:33:20
03-09-12 11:33:20:96 UTC+8 INFO [sn1] rg1-rn1: ProcessMonitor: restartProcess called, totalRestarts is 2, restartCount is -1
03-09-12 11:33:20:96 UTC+8 INFO [sn1] rg1-rn1: ProcessMonitor: startProcess
03-09-12 11:33:21:47 UTC+8 INFO [sn1] rg1-rn1: ProcessMonitor: Java HotSpot(TM) Server VM warning: Failed to reserve shared memory (errno = 12).
03-09-12 11:33:21:46 UTC+8 INFO [rg1-rn1] RepNodeStateUpdateThread started
03-09-12 11:33:21:52 UTC+8 INFO [rg1-rn1] Partition db cache mode: EVICT_LN
03-09-12 11:33:21:63 UTC+8 INFO [rg1-rn1] JVM Runtime maxMemory (bytes): 99811328
03-09-12 11:33:21:64 UTC+8 INFO [rg1-rn1] Non-default JE properties for environment: {je.rep.vlsn.logCacheSize=128, je.rep.envUnknownStateTimeout=10 s, je.tree.maxDelta=10, je.log.faultReadSize=4096, je.rep.vlsn.distance=1000000, je.env.runEvictor=true, je.cleaner.minUtilization=40, je.rep.replicaAckTimeout=5 s, je.cleaner.threads=2, je.evictor.lruOnly=false, je.rep.replayMaxOpenDbHandles=100, je.log.writeQueueSize=2097152, je.evictor.criticalPercentage=105, je.log.iteratorReadSize=1048576, je.log.numBuffers=16, je.nodeMaxEntries=128, je.evictor.nodesPerScan=200, je.rep.preHeartbeatTimeoutMs=5000000000, je.checkpointer.bytesInterval=200000000, je.cleaner.lazyMigration=false, je.txn.durability=write_no_sync,write_no_sync,simple_majority, je.cleaner.readSize=1048576, je.rep.txnRollbackLimit=10, je.log.fileCacheSize=2000, je.log.fileMax=1073741824}
03-09-12 11:33:21:69 UTC+8 INFO [rg1-rn1] Starting RepNodeService
03-09-12 11:33:21:82 UTC+8 INFO [rg1-rn1] Starting MonitorAgent
03-09-12 11:33:21:85 UTC+8 INFO [rg1-rn1] RepNodeAdmin registered
03-09-12 11:33:22:24 UTC+8 INFO [rg1-rn1] JE recovery: FIND_END_OF_LOG
03-09-12 11:33:26:97 UTC+8 INFO [rg1-rn1] JE recovery: FIND_LAST_CKPT
03-09-12 11:33:26:98 UTC+8 INFO [rg1-rn1] JE recovery: READ_DBMAP_INFO
03-09-12 11:33:41:49 UTC+8 INFO [rg1-rn1] JE recovery: UNDO_DBMAP_RECORDS
03-09-12 11:33:43:51 UTC+8 INFO [admin1] [rg1-rn1] Service status: STARTING 03-09-12 11:33:43
03-09-12 11:34:18:67 UTC+8 INFO [sn1] sn1: Shutdown starting, stopping services
03-09-12 11:34:18:67 UTC+8 INFO [sn1] Service status changed from RUNNING to STOPPING
03-09-12 11:34:23:44 UTC+8 INFO [admin1] [sn1] Service status: STOPPING 03-09-12 11:34:18
03-09-12 11:35:04:24 UTC+8 SEVERE [sn1] Cannot get RepNodeAdmin handle from Registry: rg1-rn1: Error unmarshaling return; nested exception is:
java.io.EOFException
03-09-12 11:35:04:24 UTC+8 WARNING [sn1] rg1-rn1: Exception stopping RepNode: Cannot get RepNodeAdmin handle from Registry: rg1-rn1: Error unmarshaling return; nested exception is:
java.io.EOFException; nested exception is:
java.rmi.UnmarshalException: Error unmarshaling return; nested exception is:
java.io.EOFException
03-09-12 11:35:04:24 UTC+8 INFO [sn1] rg1-rn1: ProcessMonitor: stopProcess
03-09-12 11:35:05:66 UTC+8 INFO [sn1] rg1-rn1: ProcessMonitor: IOThread exception: Stream closed
03-09-12 11:35:05:66 UTC+8 INFO [sn1] rg1-rn1: ProcessMonitor: IOThread exiting
03-09-12 11:36:05:16 UTC+8 INFO [admin1] [admin1] rg1-rn1: Service status: UNREACHABLE 03-09-12 11:36:05
  • 1. Re: Exception in thread "RMI
    Charles Lamb Pro
    Currently Being Moderated
    It seems that you need to increase the amount of memory you are allocating.

    Charles Lamb
  • 2. Re: Exception in thread "RMI
    Linda Lee Journeyer
    Currently Being Moderated
    Note that you also have this message in your log:

    03-09-12 11:33:21:47 UTC+8 INFO [sn1] rg1-rn1: ProcessMonitor: Java HotSpot(TM) Server VM warning: Failed to reserve shared memory (errno = 12).

    which implies that something else might be wrong with the JVM running on that machine. While 90M is a pretty small cache for a NoSQL node, the node should still come up, unless (a) the node has been operating with a much larger cache in the past, and the node was not shut down cleanly before or (b) The JVM heap is not really able to access all its memory.
  • 3. Re: Exception in thread "RMI
    915071 Newbie
    Currently Being Moderated
    Hello,
    I assign 90M which is computed by "java -jar je.jar DbCacheSize -records 1000000 -key 10 -data 20", and 90M is more than the max Bytes for Cache Size.

    bty, I want to know what is the difference between "Database Cache Size" and "internal Node Usage by Btree Level" ? How do I set the value for both?

    Edited by: foreversunyao on 2012-3-14 上午12:55
  • 4. Re: Exception in thread "RMI
    Charles Lamb Pro
    Currently Being Moderated
    foreversunyao wrote:
    Hello,
    I assign 90M which is computed by "java -jar je.jar DbCacheSize -records 1000000 -key 10 -data 20", and 90M is more than the max Bytes for Cache Size.

    bty, I want to know what is the difference between "Database Cache Size" and "internal Node Usage by Btree Level" ? How do I set the value for both?
    Could you please send any policy parameters that you are setting in your configuration step?

    The Internal Node Usage by Btree Level is an estimate of how much space the Internal Nodes of the B-tree will take in the cache. It is not a parameter -- only an estimate given by DbCacheSize. The Database Cache Size can be set with

    set policy "cacheSize=nnnnnn"

    in the NoSQL Database configuration.

    Charles
  • 5. Re: Exception in thread "RMI
    Charles Lamb Pro
    Currently Being Moderated
    Also, do you still have this exception in your logs?

    03-09-12 11:33:21:47 UTC+8 INFO [sn1] rg1-rn1: ProcessMonitor: Java HotSpot(TM) Server VM warning: Failed to reserve shared memory (errno = 12).
  • 6. Re: Exception in thread "RMI
    915071 Newbie
    Currently Being Moderated
    Hello,
    My policy information is belowed:
    kv-> show policy
    collectEnvStats=false
    maxTrackedLatency=1000 MILLISECONDS
    serviceLogFileCount=20
    adminLogFileCount=20
    latencyCeiling=0
    configProperties=
    adminLogFileLimit=4000000
    collectorPollPeriod=20 SECONDS
    serviceLogFileLimit=2000000
    throughputFloor=0
    javaMiscParams=-server -d32 -Xms150m -Xmx200m
    loggingConfigProps=
    eventExpiryAge=30 DAYS
    cacheSize=100000000
    statsInterval=10 SECONDS

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points