6 Replies Latest reply: Apr 20, 2012 1:18 PM by 785101 RSS

    Production site RAC redundant architecture deployment

    hitgon
      We need the advise from Oracle Experts regarding our production site RAC redundant architecture deployment

      For some business constraint, We have only 2-NIC Available for configure the redundant RAC Deployment

      So are planed to bonding the available 2-NIC and after NIC bonding we got only one bonding interface of 1gbps speed.

      We have configure the public and private ip address in that available single bonding interface.

      network configuration
      #bond0 (public network)
      10.106.1.246 rac1
      10.106.1.247 rac2

      #bond0:1(vip network 10.106.1.251)
      10.106.1.251 rac1-vip
      10.106.1.252 rac2-vip

      #bond0:11(private network 10.10.0.1)
      10.10.0.1 rac1-priv
      10.10.0.2 rac2-priv

      #bond0:2(scan ip)
      10.106.1.244 rac-scan

      Oracle Setup Details

      Oracle11g r2 RAC/GRID Software
      RAC/GRID: 2-Node Database Cluster
      Database Storage: ASM

      Please find the review of our Hardware and Software

      Hardware:
      IBM Servers and SAN Storage
      NIC Speed: 1Gbps
      Software: Redhat Enterprise Linux5.5 64bit

      Application Behavior:
      1.) High insert/update/delete/select on single table.
      2.) multiple session connect and disconnect.

      Can you please conform that above architecture will support in production site?
      what is the advantages and disadvantages with above architecture in production site?
      Can you please suggest the right way of deployment the redundant RAC deployment in production site?
        • 1. Re: Production site RAC redundant architecture deployment
          Sebastian Solbach -Dba Community-Oracle
          Hi,

          This is not supported. The private network has to be private: It may only share networks on switches via. VLAN if a Quality of Service can be guaranteed. I
          But even then best practice is it to keep it separate.

          I also doubt that you will be able to install GI, since it does not allow you to specify a :X interface. It only accepts bond0 or eth0.

          Here a short explanation, why this is not a good idea: You think by this kind of bonding you will get higher availability. But the opposite will be the case: If the public network gets a high load (due to high load (e.g. insets), it probably will saturate your network bandwith. As a result the private interconnect cannot communicate anymore: Leading a.) To slow performance and especially b.) to node evictions.

          So even if best practice states to bond network interfaces for public and private, that if you cannot guarantee "private" network it is better to go with a single card for public and a single for interconnect.

          Regards
          Sebastian
          • 2. Re: Production site RAC redundant architecture deployment
            Jrstern-Oracle
            Why can't you get another NIC for the server? RAC requires 2 NICs, and if you are using NFS, then preferably 3. If you want redundancy, you need more. If you can't get another NIC (for space or $$ reasons), then you won't have redundancy.
            • 3. Re: Production site RAC redundant architecture deployment
              hitgon
              Hello ssolbach

              Thanks for Response,

              We have 2-NIC which have same speed of 1Gbps and we had implemented bonding on that so we got single bond interface for rac installation.

              We have install and deploy the Oracle11g r2 RAC/GRID using single bonded NIC and that is install and working but now our this site is going to
              to live Production site.

              So we have to conform that whether it is supported and working in Production Environment ??

              Can you Plz provide the advantages and disadvantages with details examples?.

              Thanks & Regards
              Hitgon

              Edited by: hitgon on Sep 20, 2011 11:15 AM
              • 4. Re: Production site RAC redundant architecture deployment
                hitgon
                Hey,

                My Production sites will running with the mention configuration

                single bonding interface for public and private network :)

                Thanks to all gurus

                Regards
                Hitgon
                • 5. Re: Production site RAC redundant architecture deployment
                  onedbguru
                  This is one case where I would say that you are playing a very dangerous game with your production system. You asked for Expert opinions and you have been informed that this is a VERY BAD IDEA!!!! While you think that it works, don't come asking about node evictions when your bonded nics get saturated. I can say that I am an expert having installed, configured and spent time troubleshooting more than 75 clusters (2-6 nodes) on some very impressive hardware. The "big one" was 250TB on a 3 node RAC on Sun 6900's (48 dual-core x 192GB main memory with 8 NICS using SUN IPMP and 8 HBA's for SAN connectivity ). When you start having "weird" issues, Oracle will not support your configuration. You will need to fix it before they even begin troubleshooting it. Tell your manager that unless they spring for the appropriate configuration they should execute the following command: "Alter manager update Resume;" because it is not "IF" it will fail, but "WHEN" will it fail. Trust me, you and your managers have put your system in a very precarious position.
                  • 6. Re: Production site RAC redundant architecture deployment
                    785101
                    very similar background as onedbguru here and same response as onedbguru. You are asking for trouble with this setup. Single bonded nics can be prone to failure on heavy loads (I have seen many instances of flapping nics due to dropped packets on active networks or from switch problems during loads).