This discussion is archived
9 Replies Latest reply: Jan 18, 2013 6:21 AM by chpruvos RSS

How to choose the good Network Interface on a server ?

chpruvos Newbie
Currently Being Moderated
Hi all,

On server you have generally more than one ip address so ...How to choose the good Network Interface on a server for the NoSQL Node ?

Thank you for all

C.Pruvost.
  • 1. Re: How to choose the good Network Interface on a server ?
    Charles Lamb Pro
    Currently Being Moderated
    chpruvos wrote:
    Hi all,

    On server you have generally more than one ip address so ...How to choose the good Network Interface on a server for the NoSQL Node ?
    You need to consider three types of communications paths:

    (1) From the K/V Client to the backend (client/server comms)
    (2) Comms between the Rep Nodes (JE/HA protocol), and
    (3) Comms between the admin console and the admin database (which lives on the server nodes).

    In general (3) is negligible. (1) and (2) are more application dependent.

    Does this make sense?

    Charles
  • 2. Re: How to choose the good Network Interface on a server ?
    chpruvos Newbie
    Currently Being Moderated
    Hi Charles I only consider storage nodes and their replicated nodes.

    Actually I show an error in the logs saying that one of my storage node use a loopback address. On my laptop I have a network interface and a loopback address (disabled)...when starting sn1 I show that the port 5000 is only linked to the ip address 0.0.0.0 so it is not the good ip address..although i use for configuring :

    java -jar ..\lib\kvstore-2.0.23.jar makebootconfig -root root1 -port 5000 -admin 5001 -host ipaddress_of_my_laptop -harange 5010,5020

    and after

    java -jar ..\lib\kvstore-2.0.23.jar start -root root1

    If you have a sample tutorial that explains how simulate on the same laptop an Oracle NoSQL architecture running on 3 disctinct servers please give us this information.

    Thank you for all.

    Christophe.

    PS : error message

    Failures: Task 1 ERROR at 2013-01-15 07:23:19 UTC: DeploySN sn3(pruv
    ost-pc:5200): 1/DeploySN failed.: Register: Cannot mix loopback and non-loopback
    addresses in the same store. The store value is not configured to use loopback
    addresses but storage node pruvost-pc:5200 is a loopback address. (11.2.2.0.23)
    : Register: Cannot mix loopback and non-loopback addresses in the same store. T
    he store value is not configured to use loopback addresses but storage node pruv
    ost-pc:5200 is a loopback address. (11.2.2.0.23) java.lang.IllegalStateException
    : Register: Cannot mix loopback and non-loopback addresses in the same store. T
    he store value is not configured to use loopback addresses but storage node pruv
    ost-pc:5200 is a loopback address.
    at oracle.kv.impl.sna.StorageNodeAgent.register(StorageNodeAgent.java:15
    48)
    at oracle.kv.impl.sna.StorageNodeAgentImpl$2.execute(StorageNodeAgentImp
    l.java:194)
    at oracle.kv.impl.sna.StorageNodeAgentImpl$2.execute(StorageNodeAgentImp
    l.java:188)
    at oracle.kv.impl.fault.ProcessFaultHandler.execute(ProcessFaultHandler.
    java:90)
    at oracle.kv.impl.sna.StorageNodeAgentImpl.register(StorageNodeAgentImpl
    .java:186)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
    java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
    sorImpl.java:25)

    Edited by: chpruvos on Jan 14, 2013 11:24 PM

    And my ipconfig

    Windows IP Configuration


    Wireless LAN adapter Wireless Network Connection 2:

    Media State . . . . . . . . . . . : Media disconnected
    Connection-specific DNS Suffix . :

    Ethernet adapter Local Area Connection 3:

    Connection-specific DNS Suffix . :
    IPv4 Address. . . . . . . . . . . : 172.20.10.2
    Subnet Mask . . . . . . . . . . . : 255.255.255.240
    Default Gateway . . . . . . . . . : 172.20.10.1

    Ethernet adapter Local Area Connection* 24:

    Media State . . . . . . . . . . . : Media disconnected
    Connection-specific DNS Suffix . :

    Wireless LAN adapter Wireless Network Connection:

    Media State . . . . . . . . . . . : Media disconnected
    Connection-specific DNS Suffix . :

    Ethernet adapter Local Area Connection:

    Media State . . . . . . . . . . . : Media disconnected
    Connection-specific DNS Suffix . :

    and

    C:\Users\cpruvost>ping pruvost-pc

    Pinging PRUVOST-PC.fr.oracle.com [172.20.10.2] with 32 bytes of data:
    Reply from 172.20.10.2: bytes=32 time<1ms TTL=128
    Reply from 172.20.10.2: bytes=32 time<1ms TTL=128
    Reply from 172.20.10.2: bytes=32 time<1ms TTL=128
    Reply from 172.20.10.2: bytes=32 time<1ms TTL=128

    Ping statistics for 172.20.10.2:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
    Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

    Edited by: chpruvos on Jan 14, 2013 11:25 PM
  • 3. Re: How to choose the good Network Interface on a server ?
    Linda Lee Journeyer
    Currently Being Moderated
    Here's an entry from our updated (soon to be published) FAQ. Let us know if it answers your question!

    Should I ever create multiple Storage Nodes on the same node?

    It is strongly recommended that Storage Nodes (SNs) be allocated one per node in the cluster for availability and performance reasons. If the user believes that a given node has the I/O and CPU resources to host multiple Replication Nodes, the Storage Node's capacity property should be be set to the appropriate value. This way, the system can:

    - ensure that each Replication Node in a shard is hosted on a different Storage Node, reducing a shard's vulnerability to failure
    - dynamically divide up memory and other hardware resources among the Replication Nodes
    - ensures that the Replication Node which service write operations in a store are distributed evenly among the Storage Nodes, both at startup, and in the event of any failovers.

    If more than one SN is hosted on the same node, multiple SNs are lost if that node fails, and data may become inaccessible.

    You can set the capacity parameter for a Storage Node several ways:

    - when using the makebootconfig command
    - with the change-policy command
    - with the plan change-params commmand.

    In very limited situations, such as for early prototyping and experimentation, it can be useful to create multiple SNs on the same node.

    On a single machine a Storage Node is uniquely identified by its root director (KVROOT) plus a configuration file name, which defaults to "config.xml." This means you can create multiple SNs this way:

    Create a unique KVROOT for each SN. Usually, these would be on different nodes, but it's also possible to have them on a single node. For example, if you decide to put all SNs in the directory /var/kv/stores you might create and start more than one this way:

    mkdir /var/kv/stores/root1
    mkdir /var/kv/stores/root2
    mkdir /var/kv/stores/root3
    java -jar kvstore.jar makebootconfig -root /var/kv/stores/root1 -host fooHost -port 5000 -admin 5001 -harange 5005,5010 -capacity 1
    java -jar kvstore.jar makebootconfig -root /var/kv/stores/root2 -host fooHost -port 5020 -harange 5025,5030 -capacity 1
    java -jar kvstore.jar makebootconfig -root /var/kv/stores/root3 -host fooHost -port 5040 -harange 5045,5050 -capacity 1
    java -jar kvstore.jar start -root /var/kv/stores/root1
    java -jar kvstore.jar start -root /var/kv/stores/root2
    java -jar kvstore.jar start -root /var/kv/stores/root3

    Edited by: Linda Lee on Jan 15, 2013 9:59 AM
  • 4. Re: How to choose the good Network Interface on a server ?
    chpruvos Newbie
    Currently Being Moderated
    Ok I did that and after

    java -jar ..\lib\kvstore-2.0.23.jar runadmin -port 5000 -host pruvost-pc

    configure -name "mystore"

    plan deploy-datacenter -name MyDC -rf 2 -wait

    plan deploy-sn -dcname MyDC -host pruvost-pc -port 5000 -wait

    plan deploy-admin -sn sn1 -port 5001

    pool create -name MyPool

    pool join -name MyPool -sn sn1

    plan deploy-sn -dcname MyDC -host pruvost-pc -port 5020 -wait

    pool join -name MyPool -sn sn2

    plan deploy-sn -dcname MyDC -host pruvost-pc -port 5040 -wait

    pool join -name MyPool -sn sn3

    topology create -name MyStoreLayout -pool MyPool -partitions 100

    topology preview -name MyStoreLayout

    ****
    kv-> topology preview -name MyStoreLayout
    Topology transformation from current deployed topology to MyStoreLayout:
    Create 1 shard
    Create 2 RNs
    Create 100 partitions

    shard rg1
    2 new RNs : rg1-rn1 rg1-rn2
    100 new partitions
    ****

    plan deploy-topology -name MyStoreLayout -wait

    All the steps run good except the last one....Where I have error...It is like the post concerning an error on rmi but the question is what can I do ?

    kv-> plan deploy-topology -name MyStoreLayout -wait
    Executed plan 6, waiting for completion...
    Plan 6 ended with errors. Use "show plan -id 6" for more information
    kv-> show plan -id 6
    Plan Deploy Topo (6)
    State: ERROR
    Attempt number: 1
    Started: 2013-01-16 13:16:01 UTC
    Ended: 2013-01-16 13:20:04 UTC
    Plan failures:
    Failure 1: 6/NewNthRNParameters failed.: mystore:rg1-rn1:ADMIN
    Total tasks: 10
    Successful: 5
    Failed: 1
    Not started: 4

    Failures: Task 6 ERROR at 2013-01-16 13:20:04 UTC: NewNthRNParameter
    s: 6/NewNthRNParameters failed.: mystore:rg1-rn1:ADMIN: java.rmi.NotBoundExcepti
    on: mystore:rg1-rn1:ADMIN
    at sun.rmi.registry.RegistryImpl.lookup(RegistryImpl.java:106)
    at sun.rmi.registry.RegistryImpl_Skel.dispatch(Unknown Source)
    at sun.rmi.server.UnicastServerRef.oldDispatch(UnicastServerRef.java:386
    )
    at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:250)
    at sun.rmi.transport.Transport$1.run(Transport.java:159)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
    at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:5
    35)
    at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTranspor
    t.java:790)
    at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport
    .java:649)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec
    utor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
    .java:908)
    at java.lang.Thread.run(Thread.java:662)
    at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(Stream
    RemoteCall.java:255)
    at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:
    233)
  • 5. Re: How to choose the good Network Interface on a server ?
    Charles Lamb Pro
    Currently Being Moderated
    You need to look in the logs under KVROOT. One possibility is that one of the Rep Nodes did not start because of insufficient memory, but you'll need to confirm that by finding the relevant place in the logs.

    Charles
  • 6. Re: How to choose the good Network Interface on a server ?
    Linda Lee Journeyer
    Currently Being Moderated
    In particular, look at the information in this thread, on how to diagnose a startup issue: What does java.rmi.ConnectException: Connection refused to host mean? ¶
  • 7. Re: How to choose the good Network Interface on a server ?
    chpruvos Newbie
    Currently Being Moderated
    Ok I found I do not have enough memory...

    My laptop has 8 Go RAM.

    I tried to do that : java -jar ..\lib\kvstore-2.0.23.jar makebootconfig -root root1 -port 5000 -admin 5001 -host PRUVOST-PC.fr.oracle.com -harange 5005,5010 -capacity 1 -memory_mb 2048

    I was thinking that Oracle NoSQL will not use more than 2 Go ...

    but finally when starting I note in the log of the storage node : -Xms1740M, -Xmx1740M

    So 3 storage node and 3 replicated nodes --> I got error like "Could not reserve enough space for object heap"

    So please just tell me what should I do to have only jvm with 512 Mo Heap for exemple and not 1740 ?

    Thank you for all

    Christophe.
  • 8. Re: How to choose the good Network Interface on a server ?
    Linda Lee Journeyer
    Currently Being Moderated
    Christophe,

    By default, each StorageNode will confine the processes managed by that Storage Node to the amount of memory that is specified during the makebootconfig command. The sum of the processes managed by this Storage Node will use that memory_mb value as an upper bound. So in your case, you specified that each SN has 2G. The system decided that this Storage Node will host one replication node, and it allocated the Replication Node with a percentage (85%) of memory_mb, which turns out to be 1.7 G.

    It sounds like you are deploying on a single laptop, as a prototype, and all the SNs are on the same machine. Given that, all the processes created as part of the NoSQL server should use less than 3 * 2G, which should be possible in theory on your laptop. But perhaps there are other things running on the machine -- at any rate, the JVM couldn't allocate the expected 1.7G at startup.

    In this case, you can reduce the memory_mb value specified to the Storage Node. For example, if you want the Replication Node to have 512M, you would set the memory_mb value to 602M. You can do that at makebootconfig time. Or, if you wished to change memory_mb on a system after deployment, you can change that parameter on an existing Storage Node, using the plan change-parameters command, as follows:

    kv-> plan change-parameters
    Missing required argument for command: change-parameters
    Usage: plan change-parameters -service <id> | -all-rns |
    -all-admins [-dry-run]
    [-plan-name <name>] [-wait] [-noexecute] [-force]
    -params [name=value]*
    Change parameters for the specified service or all service
    instances of the same type. The -service flag allows a single instance
    to be affected. The -all-* flags can be used to change all instances
    of the service type. The parameters to change follow the -params
    flag and are separated by spaces. Parameter values with embedded spaces
    must be quoted. For example name="value with spaces". If -dry-run is
    specified the new parameters are returned without changing them.

    Use "show parameters" to see what parameters can be modified
    kv->

    But it's probably easier and less error prone if you just wipe out your install, by deleting the contents of the KVROOT directories and re-running the makebootconfig command with different parameters, so that if you want to repeat the steps, your script will be updated.

    Note that you can also override the defaults, and set the JVM heap size to a specific value, as was required in NoSQL DB release 1. But we do not recommend doing that with NoSQL DB Release 2, because setting a specific value prevents the system from doing dynamic adjustments when the topology of the store changes.

    -Linda
  • 9. Re: How to choose the good Network Interface on a server ?
    chpruvos Newbie
    Currently Being Moderated
    Thank you it runs...

    kv-> sho topology
    store=mystore numPartitions=100 sequence=107
    dc=[dc1] name=MyDC repFactor=1

    sn=[sn1] dc=dc1 PRUVOST-PC.fr.oracle.com:5000 capacity=1 RUNNING
    [rg1-rn1] RUNNING
    No performance info available
    sn=[sn2] dc=dc1 PRUVOST-PC.fr.oracle.com:5020 capacity=1 RUNNING
    [rg2-rn1] RUNNING
    No performance info available

    shard=[rg1] num partitions=50
    [rg1-rn1] sn=sn1
    shard=[rg2] num partitions=50
    [rg2-rn1] sn=sn2

    I saw 6 java processes...please explain...

    2 storage nodes
    2 replicated nodes

    and may be 2 NoSQL Agent ?

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points