1 2 Previous Next 22 Replies Latest reply on Jul 13, 2007 2:57 PM by 587954

    Oracle 10g RAC design with ASM and OCFS

      Hi all,

      I have a question about a proposed Oracle 10g Release 2 RAC design for a 2 node cluster.
      ASM can store database files but not Oracle binaries nor OCR and voting disk. As such, OCFS version 1 does not support a shared Oracle Home. We plan to use OCFS version 2 with ASM version 2 on Red Hat Linux Enteprrise Server 4 with Oracle 10g Release 2 (

      For OCFS v2, a shared Oracle home and shared OCR and voting disk are supported. My question is does the following proposed architecture make sense for OCFS v2 with ASM v2 on Red Hat Linux 4?

      Oracle 10g Release 2 on Red Hat Enterprise Linux Server 4:

      OCFS V2:
      - shared Oracle home and binaries
      - shared OCR and vdisk files
      - CRS software shared OCFS v2 filesystem
      - spfile
      - controlfiles
      - tnsnames.ora

      ASM v2 with ASMLib v2:
      Proposed ASM disk groups:

      - data_dg for application data
      - backupdg for flashback and archivelogs
      - undo_rac1dg ASM diskgroup for undo tablespace for racnode1
      - undo_rac2dg ASM diskgroup for undo tablespace for racnode2
      - redo_rac1dg ASM diskgroup to hold redo logs for racnode1
      - redo_rac2dg ASM diskgroup to hold redo logs for racnode2
      - temp1dg temp tablespace for racnode1
      - temp2dg temp tablespace for racnode2

      Does this sound like a good initial design?

      Ben Prusinski, Senior DBA
        • 1. Re: Oracle 10g RAC design with ASM and OCFS
          Hi, from my little experience it sound good.
          I am working on design of a 2 noeuds with the same Oracle components but on wonderful Windows.
          why do you choose OCFS2 for : controlfile and spfile ?
          You have a lot of physical disks for all those diskgroups ?

          • 2. Re: Oracle 10g RAC design with ASM and OCFS

            Personally, I would:

            - Avoid the shared Oracle home and go for a separate one on each machine.
            - Place the OCR location and voting disk on raw partitions to avoid the use of OCFS2.
            - Put all the database stuff in ASM, including the spfile, controlfiles and flash recovery area etc.
            - You can switch between ASM with Raw Devices and ASM with ASMLib quite easily, so you can try both ans stick with the one you prefer.



            • 3. Re: Oracle 10g RAC design with ASM and OCFS
              Pesonally I agree with Tim

              We have a 3 node cluster runnign OCFS2 with shared Oracle Home on windows right now.

              We have had 3 major outages in 18months - ALL ocfs related. Basically bug issues as well as an unknown issue with the communication at the ocfs layer where it appears that comms was taking place over the public interface for 2 nodes and private for one.

              I would definately go with non clustered ORACLE_HOME solution and then use either ASM/raw for DB.

              • 4. Re: Oracle 10g RAC design with ASM and OCFS
                Thanks John and Tim for your operational ( non marketing ;-) ) feedback.
                For my setup I won't use shared ORACLE_HOME and CRS_HOME.
                I need to use OCFS2 for FLASH_RECOVERY_AREA, OCR and Voting disk.
                Any advice ?

                • 5. Re: Oracle 10g RAC design with ASM and OCFS

                  Well, yo don't need to use OCFS2. You can put the OCR location and Voting Disk onto raw partitions on either separate drives, or separate partitions on the same drives. There is not advantage to useingOCFS2 for this as far as I can see.

                  Regarding the flash recovery area, this can be assigned to an ASM group, along with the archivelog destination:


                  So there is no erason to use OCFS2 unless you want to :)


                  • 6. Re: Oracle 10g RAC design with ASM and OCFS
                    Hi Tim, yes I know, but I need to use OCFS2 for the flash_recovery_area because here I can't use RMAN to backup to tape, it is not a technical issue but a political-psychological issue, the Tape robot manager don't want me to take his work ;-)
                    So I need a shared space for my disk based backup where the OS backup tools can see the files to put them to tape, so not ASM or RAW.

                    I will see if I can modify my setup to put OCR and VotingDisk on RAW device (same RAID1 disk) and keep space for an OCFS2 partition.

                    • 7. Re: Oracle 10g RAC design with ASM and OCFS
                      Backup without RMAN... Aaaaarrrrgggghhhh! Just kidding :)

                      Wait a minute. Does that mean you aren't using ASM? If you were planning to use ASM then you have to use RMAN, since it is the only supported backup tool:



                      • 8. Re: Oracle 10g RAC design with ASM and OCFS
                        Tim, really not, I use RMAN everywhere I can ;-)
                        I will have an ASM disk group for databases files.
                        The only thing I can't do is using RMAN to backup to tape, I will use RMAN using disk-based backup in the FLASH_RECOVERY_AREA but I also need to put theses files to tapes.
                        So the FLASH_RECOVERY_AREA need to be on a file system that can be used by Windows ( to run Netback up) and also want to "simplify recovery " by having all instances able to find any archived logs.
                        That is why (I think) I need an OCFS partition.
                        • 9. Re: Oracle 10g RAC design with ASM and OCFS

                          You can go either way with this.

                          You could put your flash recovery area in ASM, the do disk based RMAN backups to a filesystem outside the flash recovery area, either OCFS2 or just an NFS mount. Alternatively, you could stick with your plan. Either sounds fine.

                          I assume Netbackup doesn't integate with RMAN then. Bummer!


                          • 10. Re: Oracle 10g RAC design with ASM and OCFS
                            OK Tim, thanks for advices.
                            I think Netbackup can be integrated with RMAN but I don't want to loose time on this (political).

                            To summarize:

                            ORACLE_HOME and CRS_HOME on each node (RAID1 and NTFS)

                            Shared storage:
                            Disk1 and disk 2: RAID1: - Raw partition 1 for OCR
                            - Raw partition 2 for VotingDisk
                            - OCFS for FLASH_RECOVERY_AREA

                            Disk3, disk4 and disk5: RAID 0 - Raw with ASM redundancy normal 1 diskgroup for database files.

                            This is a running project here, will start testing the design on VMware and then go for production setup.

                            • 11. Re: Oracle 10g RAC design with ASM and OCFS

                              Just a thought. Oracle recommend multipel locations for OCR and Voting disk. I suppose this doesn't matter since you have external redundency (RAID1) for these partitions. :)


                              • 12. Re: Oracle 10g RAC design with ASM and OCFS
                                yes you are right, RAID1 will protect thoses files, and I will also backup them.

                                • 13. Re: Oracle 10g RAC design with ASM and OCFS
                                  Hi all,

                                  First, I think it would be better to use hw RAID 1 for datafiles and use ASM to distribute data across disk group. If your hw support RAID 0+1, you can join multi disks into a large partition. Then the setup will be more simple and the performance of hw RAID would be better.

                                  For the RMAN backup, you can use RMAN backup to internal disks in each box to avoid using OCFS. It is because the traditional OS filesystem is more reliable.

                                  • 14. Re: Oracle 10g RAC design with ASM and OCFS
                                    Hi Marco, thanks for remarks.
                                    I want to use ASM redundancy for the provisioning and rebalance features.
                                    If I use hardware RAID 1 , when my volume will become full I can't add another disk to the RAID 1 array, I will have to create a second RAID1 and aggregate at OS level ( I am using Windows).

                                    For RMAN, if I use internal disk ( you mean local node disks ?) as destination, If I loose a node and also loose my RMAN disk-based backups.

                                    1 2 Previous Next