This discussion is archived
1 2 Previous Next 18 Replies Latest reply: Jan 21, 2013 3:39 AM by AnkitAshokAggarwal RSS

Is Interconnect mandatory in ORACLE RAC??

AnkitAshokAggarwal Explorer
Currently Being Moderated
The private interconnect is the physical construct that allows inter-node communication or Cache Fusion communication.

because of high speed interconnect existing in the cluster between each instance. Each of instance is connected to other instance using a high-speed interconnect. This makes it possible to share the memory between 2 or more servers. Previously only datafile sharing was possible, now because of interconnect, even the cache memory can be shared.

also private IP range for cluster services cannot be used at public IP range domain due to security purpose.

so in my opinion, interconnect is must in RAC. Please share your opinion? thanks
  • 1. Re: Is Interconnect mandatory in ORACLE RAC??
    krishan Explorer
    Currently Being Moderated
    However cross cables can be used between 2 nodes but Interconnect gives more flexibility. Cross cables have technical limitations, not so robust and reliable and can't be used if your clusterware has more than 2 nodes.

    HTH
  • 2. Re: Is Interconnect mandatory in ORACLE RAC??
    Levi-Pereira Guru
    Currently Being Moderated
    Previously only datafile sharing was possible, now because of interconnect, even the cache memory can be shared.
    Only datafile sharing??? I don't get it.

    What age you are talking about? Since Oracle8i (1999), Cache Fusion optimizes read/write concurrency by using the interconnect to directly transfer data blocks among instances. This eliminates I/O and reduces delays for block reads to the speed of the interprocess communication (IPC) and the interconnecting network.

    Docs from 2000...
    The ability to spread instances across different machines provides you with scalability. Many hardware vendors offer machines that have upper limits on both memory and numbers of processors. With parallel server, though, if you have a machine with a 2-gigabyte upper limit on memory and your memory requirements grow beyond the 2 gigabytes, you can split your users across two machines. Each machine will have its own instance, but all users will share the same database.

    Lets move to 2012 ... forget about the past.
    See this :
    The Infiniband roadmap shows that the NDR (Next Data Rate) will scale to 320Gb/s.
    {message:id=10429061}
    Is Interconnect mandatory in ORACLE RAC??
    Talking about SHARED WORLD always there Networks. From long time ago.. YES
  • 3. Re: Is Interconnect mandatory in ORACLE RAC??
    Levi-Pereira Guru
    Currently Being Moderated
    krishan wrote:
    However cross cables can be used between 2 nodes but Interconnect gives more flexibility. Cross cables have technical limitations, not so robust and reliable and can't be used if your clusterware has more than 2 nodes.
    Cross-Over Cables is NOT supported on ORACLE RAC.
  • 4. Re: Is Interconnect mandatory in ORACLE RAC??
    BillyVerreynne Oracle ACE
    Currently Being Moderated
    Levi Pereira wrote:
    krishan wrote:
    However cross cables can be used between 2 nodes but Interconnect gives more flexibility. Cross cables have technical limitations, not so robust and reliable and can't be used if your clusterware has more than 2 nodes.
    Cross-Over Cables is NOT supported on ORACLE RAC.
    Want to add to Levi's statement.

    A fast and low latency Interconnect is one of the most critical features to make RAC perform and scale. You get that wrong, and your RAC will have performance issues, your RAC will not scale, your RAC will have stability problems. Guaranteed.

    It is like putting normal tires on a F1GP car and expecting the car to take a corner at over 200km/h. Does not work that way.

    The predominant Interconnect technology used by the fastest 500 computer clusters on this planet is Infiniband. The Interconnect used by Oracle's Exadata Database Machine, the fastest and most scalable RAC platform available, is Infiniband.

    If you have a 3+ node RAC, and you are not using Infiniband... then you need to ask yourself some very serious questions as to why you are limiting your RAC's performance and scalability.
  • 5. Re: Is Interconnect mandatory in ORACLE RAC??
    BillyVerreynne Oracle ACE
    Currently Being Moderated
    Ankit Ashok Aggarwal wrote:

    so in my opinion, interconnect is must in RAC. Please share your opinion? thanks
    No opinion. Fact. Interconnect is mandatory for RAC.

    Of course, there are a number of ways to implement an Interconnect.

    From the silly - like using cross-over Ethernet cables. To the dumb and stupid - like using a shared 100Mb Ethernet switch and subnet.

    The two most critical components to RAC are the Interconnect and Cluster/Shared Storage. Get this wrong and hefty penalty will be paid for that.
  • 6. Re: Is Interconnect mandatory in ORACLE RAC??
    AnkitAshokAggarwal Explorer
    Currently Being Moderated
    Solution -

    RAC is a shared disk clustered databases: every instance in the cluster has equal access to the database’s data on disk. This is in contrast to the shared nothing architecture employed by other RDBMS clusters. In a shared nothing architecture, each instance is responsible for a certain subset of data. Whenever a session needs that data, then the appropriate instance must be involved in serving up the data.
    The main challenge in the shared disk architecture is to establish a global memory cache across all the instances in the cluster: otherwise the clustered database becomes IO bound. Oracle establishes this shared cache via a high-speed private network referred to as the cluster interconnect.
    All the instances in a RAC cluster share access to datafiles on shared disk, though each have private redo logs and undo segments. Each instance has its own SGA and background processes and each session that connects to the cluster database connects to a specific instance in the cluster.
  • 7. Re: Is Interconnect mandatory in ORACLE RAC??
    LaserSoft Journeyer
    Currently Being Moderated
    Hi

    Without Interconnect, We cannot implement HA and features of RAC. Interconnect is mandatory for RAC.

    1. Each node in an Oracle RAC cluster must have at least one private network, also referred to as "cluster interconnect".
    2. RAC is using the private interconnect for synchronize memory regions (SGAs) of multiple nodes
    *3. Through Interconnect only "entire data blocks are shipped across the nodes"*.
    *4. Oracle ships database block buffers through broken down to packets of MTU sizes for across the nodes.*

    Thanks
    LaserSoft
  • 8. Re: Is Interconnect mandatory in ORACLE RAC??
    KODS Newbie
    Currently Being Moderated
    In a two node cluster environment we can use cross-over cable. I implemented the same development environment. Where as this is not possible for more than two nodes.
  • 9. Re: Is Interconnect mandatory in ORACLE RAC??
    BillyVerreynne Oracle ACE
    Currently Being Moderated
    Support Note +RAC: Frequently Asked Questions [ID 220970.1]+

    >
    Is crossover cable supported as an interconnect with RAC on any platform ?

    <b>NO. CROSS OVER CABLES ARE NOT SUPPORTED. </b>
    >

    Oracle's emphasis using bold and upper case. Not mine.
  • 10. Re: Is Interconnect mandatory in ORACLE RAC??
    Victor Armbrust Oracle ACE
    Currently Being Moderated
    Hi all

    I made an article in my personal Blog with best practives about Oracle RAC interconnect.
    I'm posting the English Version (since the original one is in portuguese)

    Sorry if the gramatical is not ok:

    http://www.google.com/translate?u=http%3A%2F%2Fvictor-dba.blogspot.com.br%2F2012%2F05%2Fprincipais-recomendacoes-para.html&langpair=pt%7cen&en=pt&en=utf8


    Holpe it helps

    Victor
  • 11. Re: Is Interconnect mandatory in ORACLE RAC??
    BillyVerreynne Oracle ACE
    Currently Being Moderated
    Victor Armbrust wrote:

    I made an article in my personal Blog with best practives about Oracle RAC interconnect.
    Victor, if the aim is to describe best Grid/RAC Interconnect practises then IMO:
    - 100Mb Ethernet is unacceptable and 1Gb Ethernet the minimum
    - 40Gb (QDR) Infiniband is recommended for clusters larger than 3+ nodes
    - RDS and not UDP as Interconnect protocol is recommended if Infiniband is used
    - dual and dedicated Interconnect switches are needed (no shared switches, no VLANs, etc)
    - bonded Interconnect interface running via dual ports to dual private switches

    One only needs to look at Oracle's Database Machine and its Interconnect to see what the best practises are for a highly efficient and scalable Interconnect.
  • 12. Re: Is Interconnect mandatory in ORACLE RAC??
    Victor Armbrust Oracle ACE
    Currently Being Moderated
    Billy

    I missed to comment: This one is not related to EXA... Since EXADATA has his own configuration (IB, RDS, iDB, etc.etc.)
    The focus was only on conventional RAC interconnect configuration. However your observations were well applied. Thank you

    Victor
  • 13. Re: Is Interconnect mandatory in ORACLE RAC??
    BillyVerreynne Oracle ACE
    Currently Being Moderated
    Victor, I'm saying that Oracle's Database Machine Interconnect simply shows the most optimal Interconnect for RAC. For "conventional" RAC too.

    We have been using Infiniband since 2005 for RAC, long before Database Machine. It has been a "certified/supported" h/w architecture by Oracle prior to Database Machine. We are still using Infiniband today on all our clusters (from 4 nodes to 12 nodes) - and we do not have Database Machine/Exadata.

    Infiniband is also the fastest growing Interconnect architecture and has a bigger share amongst the 500 fastest clusters in the world, than Gig Ethernet.

    Infiniband is also not more expensive - in fact, in our experience it has been cheaper than using Gig Ethernet of similar speeds.

    So recommending Gig Ethernet as Interconnect for Oracle RAC is IMO a very poor and shoddy recommendation. Infiniband is superior in every aspect. At the same, or lower, cost. As a supported configuration by Oracle for RAC ("conventional" or otherwise).
  • 14. Re: Is Interconnect mandatory in ORACLE RAC??
    Victor Armbrust Oracle ACE
    Currently Being Moderated
    Hi Billy

    Thank you for sharing your experience and recommendations. IB is a better solution for sure.

    Victor
1 2 Previous Next

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points