This discussion is archived
6 Replies Latest reply: Feb 9, 2012 6:58 PM by 915071 RSS

Exception in thread "main" java.rmi.NotBoundException: commandService

915071 Newbie
Currently Being Moderated
Hello,
I want to deploy two SNs on one machine, and I chose different port(5030,5031), when I run "runadmin", the exceptions happened.

[sunyao@datanode1:~/oracle_nosql/tools]$ jps -m
25392 Jps -m
25145 kvstore-1.2.123.jar start -root KVROOT_2
23099 kvstore-1.2.123.jar start -root KVROOT
23265 ManagedService -root KVROOT/mystore/sn2 -store mystore -class RepNode -service rg1-rn2
25220 ManagedService -root KVROOT_2/mystore2/sn1 -store mystore2 -class Admin -service admin1
[sunyao@datanode1:~/oracle_nosql/tools]$ java -jar kv-1.2.123/lib/kvstore-1.2.123.jar ping -port 5030 -host datanode1
SNA at hostname: datanode1 registry port: 5030 has no available Admins or RNs registered.
[sunyao@datanode1:~/oracle_nosql/tools]$ java -jar kv-1.2.123/lib/kvstore-1.2.123.jar runadmin -port 5030 -host datanode1
Exception in thread "main" java.rmi.NotBoundException: commandService
at sun.rmi.registry.RegistryImpl.lookup(RegistryImpl.java:106)
at sun.rmi.registry.RegistryImpl_Skel.dispatch(Unknown Source)
at sun.rmi.server.UnicastServerRef.oldDispatch(UnicastServerRef.java:386)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:250)
at sun.rmi.transport.Transport$1.run(Transport.java:159)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:255)
at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:233)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:359)
at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
at oracle.kv.impl.util.registry.RegistryUtils.getAdmin(RegistryUtils.java:197)
at oracle.kv.impl.admin.client.KVAdmin.connect(KVAdmin.java:883)
at oracle.kv.impl.admin.client.KVAdmin.main(KVAdmin.java:2131)
at oracle.kv.impl.util.KVStoreMain$6.run(KVStoreMain.java:188)
at oracle.kv.impl.util.KVStoreMain.main(KVStoreMain.java:319)
  • 1. Re: Exception in thread "main" java.rmi.NotBoundException: commandService
    guy.k.hillyer Newbie
    Currently Being Moderated
    This error message suggests that while you have a StorageNode listening on port 5030, there is no Admin associated with that StorageNode. Guessing that the configuration in KVROOT uses port number 5030, and KVROOT_2 uses port number 5031, your Admin is probably available on port 5031. I can't tell much more without knowing the contents of the config.xml files.

    It seems unusual for the two storage nodes to be using consecutive port numbers, since you have to allocate several ports for each SN, but it's not impossible to have a configuration like that. I would double-check to be sure that the two storage nodes are using distinct sets of ports.
  • 2. Re: Exception in thread "main" java.rmi.NotBoundException: commandService
    Linda Lee Journeyer
    Currently Being Moderated
    In addition to Guy's comments, you may want to review the FAQ, at http://www.oracle.com/technetwork/database/nosqldb/learnmore/nosqldb-faq-518364.html. In particular, the entries:

    - explaining the use of ports in NoSQL DB: http://www.oracle.com/technetwork/database/nosqldb/learnmore/nosqldb-faq-518364.html#WhatarealltheseportsWhendoIusewhichone
    - how to create multiple SNs on one machine: http://www.oracle.com/technetwork/database/nosqldb/learnmore/nosqldb-faq-518364.html#HowtocreatemultipleStorageNodesonthesamemachine

    may be useful.

    You may also want to try the java -jar kvstore.jar ping -host <hostname> -port <port> on various nodes to get a sense of what you've managed to deploy, and which services are up.
  • 3. Re: Exception in thread "main" java.rmi.NotBoundException: commandService
    915071 Newbie
    Currently Being Moderated
    hi ,

    I have started two admin service in one group:
    but one admin service can't access from 5001, and the log is below:
    [sunyao@datanode1:~/oracle_nosql/tools/KVROOT/mystore/log]$ cat admin2_0.log
    02-09-12 17:48:46:696 CST INFO [admin2] Starting service process: admin2, Java command line arguments: [-XX:+DisableExplicitGC, -Xms96M, -Xmx96M]
    02-09-12 17:48:46:720 CST INFO [admin2] Starting AdminService
    02-09-12 17:48:46:881 CST INFO [admin2] Initializing Admin for store: mystore
    02-09-12 17:48:46:883 CST INFO [admin2] Created new admin environment dir: KVROOT/mystore/sn2/admin2/env
    02-09-12 17:48:46:884 CST INFO [admin2] JVM Runtime maxMemory (bytes): 96468992
    02-09-12 17:48:46:889 CST INFO [admin2] Non-default JE properties for environment: {je.rep.vlsn.logCacheSize=128, je.rep.envUnknownStateTimeout=10 s, je.tree.maxDelta=10, je.log.faultReadSize=4096, je.rep.vlsn.distance=1000000, je.env.runEvictor=true, je.cleaner.minUtilization=40, je.rep.replicaAckTimeout=5 s, je.cleaner.threads=2, je.evictor.lruOnly=false, je.rep.replayMaxOpenDbHandles=100, je.log.writeQueueSize=2097152, je.evictor.criticalPercentage=105, je.log.iteratorReadSize=1048576, je.log.numBuffers=16, je.nodeMaxEntries=128, je.evictor.nodesPerScan=200, je.rep.preHeartbeatTimeoutMs=5000000000, je.checkpointer.bytesInterval=200000000, je.cleaner.lazyMigration=false, je.txn.durability=write_no_sync,write_no_sync,simple_majority, je.cleaner.readSize=1048576, je.rep.txnRollbackLimit=10, je.log.fileCacheSize=2000, je.log.fileMax=1073741824}
    02-09-12 17:48:48:390 CST INFO [admin2] State change event: Thu Feb 09 17:48:47 CST 2012, State: REPLICA, Master: 1
    02-09-12 17:48:48:852 CST INFO [admin2] Shutting down monitor collector
    02-09-12 17:48:48:853 CST INFO [admin2] Replicated environment handle established. Cache size: 57881395, State: REPLICA
    02-09-12 17:48:48:854 CST INFO [admin2] Starting Command service on rmi://datanode1:5000/commandService
    02-09-12 17:48:48:950 CST INFO [admin2] Starting Web service on port 5001
    02-09-12 17:48:49:183 CST INFO [admin2] jetty-7.4.0.v20110414
    02-09-12 17:48:49:350 CST INFO [admin2] Extract jar:file:/home/sunyao/oracle_nosql/tools/kv-1.2.123/lib/kvstore-1.2.123.jar!/war/KVAdminUI/ to /tmp/jetty-0.0.0.0-5001-KVAdminUI-_-any-/webapp
    02-09-12 17:48:50:65 CST INFO [admin2] NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
    02-09-12 17:48:50:120 CST INFO [admin2] started o.e.j.w.WebAppContext{,file:/tmp/jetty-0.0.0.0-5001-KVAdminUI-_-any-/webapp/},jar:file:/home/sunyao/oracle_nosql/tools/kv-1.2.123/lib/kvstore-1.2.123.jar!/war/KVAdminUI
    02-09-12 17:48:50:201 CST INFO [admin2] Started SelectChannelConnector@0.0.0.0:5001 STARTING
    02-09-12 17:48:50:203 CST INFO [admin2] Started AdminService
    02-09-12 17:49:04:408 CST INFO [admin2] Redirect to master at http://namenode:5001
    02-09-12 17:49:24:427 CST INFO [admin2] Redirect to master at http://namenode:5001
    02-09-12 17:49:28:432 CST INFO [admin2] Redirect to master at http://namenode:5001
    02-09-12 17:52:16:974 CST INFO [admin2] Redirect to master at http://namenode:5001
    02-09-12 17:52:29:844 CST INFO [admin2] Redirect to master at http://namenode:5001
    02-09-12 17:54:03:343 CST INFO [admin2] Redirect to master at http://namenode:5001
    02-09-12 18:00:48:854 CST INFO [admin2] Redirect to master at http://namenode:5001
    02-09-12 18:02:10:25 CST INFO [admin2] Redirect to master at http://namenode:5001
    02-09-12 18:02:55:400 CST INFO [admin2] Redirect to master at http://namenode:5001
    02-09-12 18:02:59:708 CST INFO [admin2] Redirect to master at http://namenode:5001
    02-09-12 18:03:02:127 CST INFO [admin2] Redirect to master at http://namenode:5001
    02-09-12 18:03:05:644 CST INFO [admin2] Redirect to master at http://namenode:5001

    namenode is the hostname of 192.168.7.200, and I can access the admin service from http://192.168.7.200:5001
    and the topology is
    Pinging components of store mystore based upon topology sequence #612
    mystore comprises 300 partitions and 3 Storage Nodes
    Storage Node [sn1] on namenode:5000 Datacenter: Boston [dc1] Status: RUNNING Ver: 11gR2.1.2.123
    Rep Node [rg1-rn1] Status: RUNNING,REPLICA at sequence number: 625 haPort: 5011
    Storage Node [sn2] on datanode1:5000 Datacenter: Boston [dc1] Status: RUNNING Ver: 11gR2.1.2.123
    Rep Node [rg1-rn2] Status: RUNNING,REPLICA at sequence number: 625 haPort: 5010
    Storage Node [sn3] on datanode2:5000 Datacenter: Boston [dc1] Status: RUNNING Ver: 11gR2.1.2.123
    Rep Node [rg1-rn3] Status: RUNNING,MASTER at sequence number: 625 haPort: 5010

    Edited by: 912068 on 2012-2-9 上午2:06


    and when I shutdown namenode SN and admin service , and access 185:5001, the exceptions are belowed:

    Error
    Dismiss
    The Admin instance is unable to service this request, as it is not the master Admin instance. Furthermore, its state is such that it does not know the master Admin's forwarding address.

    Edited by: 912068 on 2012-2-9 上午2:24
  • 4. Re: Exception in thread "main" java.rmi.NotBoundException: commandService
    guy.k.hillyer Newbie
    Currently Being Moderated
    If I understand you correctly, what you have described is normal behavior. It seems you have two Admins running. The one on namenode is the master, while the one on datanode1 is a replica. The Admin on datanode1 cannot service requests, because it is not the master. When runadmin tries to connect to the Admin on datanode1, the connection is automatically forwarded to the master Admin on namenode. That is shown by the log messages that say "Redirect to master".

    When you shut down namenode, the Admin mastership cannot fail over to datanode1, because it can't form a majority. That is shown by the message about "The Admin instance is unable to service this request, as it is not the master Admin instance. Furthermore..."

    In the FAQ, we recommend that you deploy at least three Admin replicas. This is the reason for that recommendation. If you have three replicas, and the master goes down, the remaining two can form a majority and choose a new master. If there are only two Admins, and one of them shuts down, then the remaining one cannot form a majority.

    http://www.oracle.com/technetwork/database/nosqldb/learnmore/nosqldb-faq-518364.html#HowmanyAdminservicesshouldIdeploy
  • 5. Re: Exception in thread "main" java.rmi.NotBoundException: commandService
    Charles Lamb Pro
    Currently Being Moderated
    Just to clarify, when Guy says you should have three replicas, he means 2 replicas and 1 master.

    Charles Lamb

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points