7 Replies Latest reply on Feb 14, 2013 9:32 AM by noli

    How to install RAC1 on DR site with SAN replication

      Hi experts, please share with your experiences on setting up RAC/RAC1 nodes on a remote DR site using SAN replication. Here is our environment:

      Primary site with RAC One nodes on 2 servers: p-node1, p-node2
      Standy DR Site will have another RAC One nodes: s-node1, s-node2

      Primary and DR Data/FRA are replicated by Hitachi HUR SAN replication.
      Grid Home and RDBMS Home are local, not shared, and non-replicated cross site.
      Clusterware/ASM (OCR/Voting disk) on CRS DG is independent on primary and DR -- not replicated cross site.

      Note: with SAN mirror, the DR site Data/FRA hdisks are not visible until failover (or split mirror).

      So, the plan is:

      1. Primray site: Install Grid Home, RDBMS Home, and create RAC1 database (by standard installation procedures)
      ... now Primary site has two RAC1 nodes functioning ...

      2. Next, on DR site: Install Grid Home/Clusterware and create another ASM instance on another non-replicated CRS diskgroup
      ... we have the 2nd Clusterware/ASM on DR, with CRS DG only
      ... at this point, the Data/FRA LUNs are replicated copy -- not visible to DR servers, until failover or split mirror.

      3. Next, shutdown primary site database and Clusterware. Then, either failover or split SAN mirror to Dr -- to make Data/FRA LUNs visible to DR s-node1 and s-node2 servers.

      1. Once the replicated Data/FRA LUNs are visible on DR, the DR ASM instance should be able to discover the LUNs automatically, correct?

      2. Since Data/FRA DG metadata is already in LUN header, will DR ASM be able to automatically incoorporate the diskgroups? (thus no need to do create diskgroups?)
      If not so, HOW do I add the Data/FRA DGs to DR ASM?
      ... there is no 'srvctl add diskgroup' command....

      3. After DR ASM recognize and mount Data/FRA diskgroups, will Clusterware automatically register the diskgroups? The following 2 links says it will automatically register at first mount. Is this correct?


      4. Finally, what needs to be done to register the database/instance/service to the DR Clusterware?

      5. What other steps are needed to complete DR Clusterware and RAC One installation?

        • 1. Re: How to install RAC1 on DR site with SAN replication
          Using SAN replication ... if you write a corrupt block ... you will replicate a corrupt block.

          My recommendation would be to go to a Physical Data Guard standby.
          • 2. Re: How to install RAC1 on DR site with SAN replication
            Agree, but there's infrastructure and application requirements that we are using SAN replication instead of DataGuard.

            Experts, please shed some lights here. Thanks.
            • 3. Re: How to install RAC1 on DR site with SAN replication
              From my practise -
              Hardware Replication -> Bad Solution
              Standby -> Best Solution

              Can you open DB if her state inconsistent(primary storage crash -> last block not replicated) - NO!!!

              Standby cost chiper than License on Hardware Replica!
              • 4. Re: How to install RAC1 on DR site with SAN replication
                Thanks to all your input. We were able to complete the installation, as follows:

                - After primary RAC nodes are installed; srvctl stop database; srvctl stop diskgroups (replicated LUNs).
                - SA mount the LUNs on standby site (split mirror), then, we installed Grid Home and RDBMS Home.
                - We were able to mount the LUNs on standby ASM. ASM automatically recognized the LUNs as MEMBER.

                Afterward, we have to run these 2 commands to register database and service:

                srvctl add database -d xxx -o $OHOME -p +DATA/xxx/spfilepocsim.ora -n xxx -c RACONENODE -e snode1,snode2 -i xxx -a "DATA,FRA,ARCH,REDO1,REDO2"

                srvctl add service -s SERVICE_NAME -d xxx -P BASIC

                (I found the above 2 commands from DB creation scripts automatically generated by DBCA; the trick is that we have to add REDO1,REDO2 to the -a option, as the generated commands by DBCA did not include REDO1/REDO2. Make sure you have all replicated diskgroups in the srvctl add database -a dg_list.)

                I will bring up the issue to our architect when primary SAN crashes and last DB block is not replicated.

                • 5. Re: How to install RAC1 on DR site with SAN replication

                  We are having similar issue here. May I know if you encounter problem in the DR site that the ASM will try to access the disk when it is having replication (end up it come with I/O errors)? Do you have CRS/Grid on the standby site in your design?
                  • 6. Re: How to install RAC1 on DR site with SAN replication
                    Hi All,

                    We are planning to do DR testing, next week with similar setup, need help , if somebody has step by step doc. please share with me

                    Thanks in advance.

                    NIARV SHAH
                    +91 7795020106
                    • 7. Re: How to install RAC1 on DR site with SAN replication
                      hi, we have a similar scenario.

                      From our Data Center we have a 2 node RAC and from our DR site we have a AIX server that also replicate the storage of our RAC from the data center. scenario will be, We will Install A single Instance and used the Replicate Copy of the storage of RAC as storage. Can you give me some advice on how can i make use of the replicate storage as the storage of the single instance? Is installing the Grid infrastructure and the RDBMS and copy the PFILE from the PRIMARY rac can make our Single Instance up?

                      Thanks in advance.