5 Replies Latest reply: Dec 10, 2012 2:26 AM by 940750 RSS

    Installing Oracle NoSQL 11gR2.2.0.23

    940750
      hello,

      I am trying to install latest version of oracle nosql I am having problems when I try to deploy the topology. When I try to do it i get following error:

      Plan Deploy Topo (6)
      State: ERROR
      Attempt number: 1
      Started: 2012-12-03 10:24:06 UTC
      Ended: 2012-12-03 10:26:08 UTC
      Plan failures:
      Failure 1: 7/UpdateNthRNHelperHost failed.: Connection refused to host: ubuntu642a; nested exception is:
      java.net.ConnectException: Connection refused
      Total tasks: 24
      Successful: 6
      Failed: 1
      Not started: 17

      Failures: Task 7 ERROR at 2012-12-03 10:26:08 UTC: UpdateNthRNHelperHost: 7/UpdateNthRNHelperHost failed.: Connection refused to host: ubuntu642a; nested exception is:
      java.net.ConnectException: Connection refused: java.rmi.ConnectException: Connection refused to host: ubuntu642a; nested exception is:
      java.net.ConnectException: Connection refused
      at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:601)
      at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:198)
      at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
      at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:322)
      at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
      at oracle.kv.impl.util.registry.RegistryUtils.lookup(RegistryUtils.java:659)
      at oracle.kv.impl.util.registry.RegistryUtils.getStorageNodeAgent(RegistryUtils.java:287)
      at oracle.kv.impl.admin.plan.task.Utils.updateHelperHost(Utils.java:292)
      at oracle.kv.impl.admin.plan.task.UpdateNthRNHelperHost.doWork(UpdateNthRNHelperHost.java:66)
      at oracle.kv.impl.admin.plan.task.SingleJobTask$1.call(SingleJobTask.java:41)
      at oracle.kv.impl.admin.plan.task.SingleJobTask$1.call(SingleJobTask.java:38)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:662)
      Caused by: java.net.ConnectException: Connection refused
      at java.net.PlainSocketImpl.socketConnect(Native Method)
      at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
      at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
      at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
      at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
      at java.net.Socket.connect(Socket.java:529)
      at java.net.Socket.connect(Socket.java:478)
      at java.net.Socket.<init>(Socket.java:375)
      at java.net.Socket.<init>(Socket.java:189)
      at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:22)
      at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:128)
      at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:595)
      ... 17 more

      I also noticed that when I try to deploy the topology processes related to oracle nosql at other storage node gets killed automatically. Kindly provide help on same.

      Thanks and Regards
      Rishabh Agrawal
        • 1. Re: Installing Oracle NoSQL 11gR2.2.0.23
          940750
          Configuration is as follows:

          <config version="2">
          <component name="params" type="bootstrapParams" validate="true">
          <property name="adminHttpPort" value="5001" type="INT"/>
          <property name="hostingAdmin" value="false" type="BOOLEAN"/>
          <property name="storeName" value="mystore" type="STRING"/>
          <property name="storageNodeId" value="2" type="INT"/>
          <property name="memoryMB" value="516" type="INT"/>
          <property name="hostname" value="ubuntu642a" type="STRING"/>
          <property name="capacity" value="2" type="INT"/>
          <property name="numCPUs" value="1" type="INT"/>
          <property name="rootDir" value="/ilabs/data/kvroot/" type="STRING"/>
          <property name="haPortRange" value="5010,5020" type="STRING"/>
          <property name="registryPort" value="5000" type="INT"/>
          </component>
          <component name="mountPoints" type="bootstrapParams" validate="false">
          </component>
          </config>

          Configuration is same for all nodes except the hostname.
          • 2. Re: Installing Oracle NoSQL 11gR2.2.0.23
            Linda Lee-Oracle
            Rishabh,

            It seems that one of the nodes that you are trying to use in this deployment is not responsive. The first thing to do is to get some general context on your store by using the "show topology" and "verify" commands to examine the state of your store now. For example, use "show topology" to see what has been deployed to date:

            kv-> show topology
            store=mystore numPartitions=100 sequence=112
            dc=[dc1] name=MyDC repFactor=3

            sn=[sn1] dc=dc1 localhost:13100 capacity=2 RUNNING
            [rg1-rn1] RUNNING
            No performance info available
            [rg2-rn1] RUNNING
            No performance info available
            sn=[sn2] dc=dc1 localhost:13200 capacity=2 RUNNING
            [rg1-rn2] RUNNING
            No performance info available
            [rg2-rn2] RUNNING
            No performance info available
            sn=[sn3] dc=dc1 localhost:13300 capacity=2 RUNNING
            [rg1-rn3] RUNNING
            No performance info available
            [rg2-rn3] RUNNING
            No performance info available

            shard=[rg1] num partitions=50
            [rg1-rn1] sn=sn1
            [rg1-rn2] sn=sn2
            [rg1-rn3] sn=sn3
            shard=[rg2] num partitions=50
            [rg2-rn1] sn=sn1
            [rg2-rn2] sn=sn2
            [rg2-rn3] sn=sn3

            and use "verify" to check the state of the different components:

            kv-> verify
            Verify: starting verification of mystore based upon topology sequence #112
            100 partitions and 3 storage nodes. Version: 11.2.2.0.23 Time: 2012-12-03 16:37:44 UTC
            See localhost:KVRT1\mystore\log\mystore_{0..N}.log for progress messages
            Verify: == checking storage node sn1 ==
            Verify: Storage Node [sn1] on localhost:13100 Datacenter: MyDC [dc1] Status: RUNNING Ver: 11gR2.2.0.23 2012-11-26 07:37:01 UTC Build id: 9cbea2b69392
            Verify: Admin [admin1] Status: RUNNING
            Verify: Rep Node [rg1-rn1] Status: RUNNING,REPLICA at sequence number: 131 haPort: 13111
            Verify: Rep Node [rg2-rn1] Status: RUNNING,MASTER at sequence number: 127 haPort: 13112
            Verify: == checking storage node sn2 ==
            Verify: Storage Node [sn2] on localhost:13200 Datacenter: MyDC [dc1] Status: RUNNING Ver: 11gR2.2.0.23 2012-11-26 07:37:01 UTC Build id: 9cbea2b69392
            Verify: Rep Node [rg1-rn2] Status: RUNNING,MASTER at sequence number: 131 haPort: 13210
            Verify: Rep Node [rg2-rn2] Status: RUNNING,REPLICA at sequence number: 127 haPort: 13211
            Verify: == checking storage node sn3 ==
            Verify: Storage Node [sn3] on localhost:13300 Datacenter: MyDC [dc1] Status: RUNNING Ver: 11gR2.2.0.23 2012-11-26 07:37:01 UTC Build id: 9cbea2b69392
            Verify: Rep Node [rg1-rn3] Status: RUNNING,REPLICA at sequence number: 131 haPort: 13310
            Verify: Rep Node [rg2-rn3] Status: RUNNING,REPLICA at sequence number: 127 haPort: 13311
            Verification complete, 0 violations, 3 notes found.

            This will probably indicate to you that node ubuntu642a is not responsive. To get more information as to why that might be so, look in the aggregated storewide log, which can be found in the node that is hosting the admin service, under the KVROOT/<storename>/logs/<storename>*.log. This shows information from all the different components n the store. You will want to look at the storage node and replication node that are hosted on ubunto642a.

            For example, if ubuntu642a hosts Storage Node sn3 and Replication Node rg1-rn3, look through the <storename>.log for entries made by those components. Each log entry is prefixed with the name of the component that issued the log message. Sometimes that aggregated store has too much information, and it is easier to look at the Replication Node or Storage Node logs directly, which can be found on their host. For example, you would go to ubuntu642a to the <KVROOT>/<storename>/logs directory to find the logs for sn2 and rg1-rn3.

            Since this sounds like it is an initial deployment, you will want to review the logs to make sure that the Storage Node Agent on that node was created correctly, and that the process came up as expected, according to the installation directions. If it was alive at the time when you issued the deploy topology command, you'll want to see what caused it to become unresponsive.

            Regards,

            Linda
            • 3. Re: Installing Oracle NoSQL 11gR2.2.0.23
              940750
              Thanks Linda for elaborate response. It really helped me out. There were mainly two issues:

              1) Host name
              2) Time sync

              Thanks again
              Rishabh
              • 4. Re: Installing Oracle NoSQL 11gR2.2.0.23
                Johan Louwers
                I notice that you are running Ubuntu. I have had something like this when installing Hadoop HBase on Ubuntu. It might be related to the way Ubuntu is implementing the settings in /etc/hosts

                You might want to have a look at this blogpost I have been writing about this subject. http://johanlouwers.blogspot.nl/2012/11/hadoop-hbase-localhost-considerations.html

                Please do keep us posted.

                Regards,
                Johan Louwers.
                • 5. Re: Installing Oracle NoSQL 11gR2.2.0.23
                  940750
                  Hello Johan,

                  Yeah you were right about the /etc/hosts file. I have commented on your blog post on same.

                  Thanks and Regards
                  Rishabh Agrawal

                  Edited by: Rishabh Agrawal on Dec 10, 2012 12:25 AM