5 Replies Latest reply: Dec 20, 2012 11:46 AM by Dude! RSS

    ASM, rebalancing, 3rd party storage and performance

    980626
      Hi all, I have some questions for the storage experts and I need your opinion about:

      1. Which type of ASM is better for the rebalancing perspective (and obviously the performance perspective), but considering the availability as well: Unprotected, 2-way (normal) or 3-way (high)? I know 3-way is the worst because we have to multiply by 3 the space of the database, but has great availability. Which of these do you use in your company?

      2. If someone is using EMC/Fujitsu/Hitachi, do you use Unprotected or 2-way (normal) in ASM?

      3. Which type of rebalancing (from zero to 11) is the best to use, considering the overall performance , I mean, when you have to add a new disk, ASm will perform the rebalancing.

      4. Which is better to use: ASM Cluster File System or Oracle Cluster File System ?

      Thanks.
        • 1. Re: ASM, rebalancing, 3rd party storage and performance
          898553
          I spent quite a lot of time doing ASM tuning over the years and this is what worked best for my company on EMC CLARiiON and Symmetrix arrays, but YMMV (and yes, I'm aware some of this contradicts Oracle's claims of just throwing everything in one DG is faster):

          We created an OCR/VOTE diskgroup using 3x2GB LUNs on 'high' redundancy, then mapped out the following for diskgroups:

          DATABASE1_FRADG
          DATABASE1_ACFSDG
          DATABASE1_DATADG

          DATABASE2_FRADG
          DATABASE2_ACFSDG
          DATABASE3_ACFSDG

          etc.

          Now for each of these diskgroups, we set the redundancy as 'external'

          As far as rebalancing, we noticed that on Dell R610s, R710s, and R815s setting the rebal power at '9' made very little difference from a systems resource perspective than 1-8 so if you ever need to rebal during business hours, I'd say go with that. If you always make your changes in a change window where clients are not connecting, there's no reason not to set the power to '11'.

          For 11G databases, I can't think of a reason to not use ACFS since it's so well integrated with CRS.

          Like I said above though, do your own testing and try it out but that's what worked best for us.
          • 2. Re: ASM, rebalancing, 3rd party storage and performance
            LaserSoft
            Hi Friend,

            Good Query...


            1. Which type of ASM is better for the rebalancing perspective (and obviously the performance perspective), but considering the availability as well: Unprotected, 2-way (normal) or 3-way (high)? I know 3-way is the worst because we have to multiply by 3 the space of the database, but has great availability. Which of these do you use in your company?


            ---> If ASM is doing the mirroring, each side of the mirror has to read individually. If the mirroring is done in a storage array then there is no interface to select mirror.

            So, leave the mirroring part Storage level.


            2. If someone is using EMC/Fujitsu/Hitachi, do you use Unprotected or 2-way (normal) in ASM?

            ----> ASM is the best file system for Oracle databases on EMC storage.EMC is not a competitor to ASM. ASM compliments EMC.


            3. Which type of rebalancing (from zero to 11) is the best to use, considering the overall performance , I mean, when you have to add a new disk, ASm will perform the rebalancing.

            ---> ASM does not block I/O during rebalance.

            4. Which is better to use: ASM Cluster File System or Oracle Cluster File System ?

            ---> We Cannot compare like that. Becuase Oracle Automatic Storage Management Cluster File System (Oracle ACFS) extends Oracle Automatic Storage Management technology to support of all of your application data in both single host and cluster configurations.

            you can use OCFS for shared oracle home data files & one might want to use ASM for datafiles and OCFS for shared oracle home.

            Hope it helps.....

            Thanks
            LaserSoft
            • 3. Re: ASM, rebalancing, 3rd party storage and performance
              onedbguru
              LaserSoft wrote:
              Hi Friend,

              Good Query...
              4. Which is better to use: ASM Cluster File System or Oracle Cluster File System ?

              ---> We Cannot compare like that. Becuase Oracle Automatic Storage Management Cluster File System (Oracle ACFS) extends Oracle Automatic Storage Management technology to support of all of your application data in both single host and cluster configurations.

              you can use OCFS for shared oracle home data files & one might want to use ASM for datafiles and OCFS for shared oracle home.
              I would look at the roadmap for OCFS before making that statement. (It does not appear that it is going to be supported much longer in favor of ACFS. Having used both, I would dump OCFS at the earliest possible timeframe again, in favor of ACFS. AFGS in 11gR2 has been specifically optimized for shared Oracle Homes. The downside to using a SHARED ORACLE_HOME is that you lose the ability to do a "rolling" upgrade/patching. Rolling upgrade means: take down node 1 (everything fails to other nodes) patch it, restart it move to the next node. There is a reason RAC is considered HIGHLY AVAILABLE. Having to take everything down to do patching/upgrading defeats that purpose.
              • 4. Re: ASM, rebalancing, 3rd party storage and performance
                Billy~Verreynne
                977623 wrote:

                1. Which type of ASM is better for the rebalancing perspective (and obviously the performance perspective), but considering the availability as well: Unprotected, 2-way (normal) or 3-way (high)? I know 3-way is the worst because we have to multiply by 3 the space of the database, but has great availability. Which of these do you use in your company?
                Wrong question IMO. You do not choose normal or high redundancy based on which rebalances the fastest. You make that choice wrt to redundancy and high availability.

                The best in this respect is high redundancy. 3 storage servers. Fully redundant I/O fabric layer (3+ I/O paths from each server to each and every LUN). Which means you can loose 2 storage arrays and most of your I/O fabric layer (switches and cables). And the storage layer left will still be able to be effectively used for ASM and the RAC database will not even notice that this destruction happened.

                However, this best is not always feasible ito existing architecture, budget and so on. So that drives the requirements and needs and what the best is to meet that.
                2. If someone is using EMC/Fujitsu/Hitachi, do you use Unprotected or 2-way (normal) in ASM?
                That again depends on redundancy levels. If the SAN provides a RAID5 or RAID10 LUN, should you mirror it?

                This may makes sense if you have 2 SANS. And with each SAN implementing RAID5, you mirror (via ASM) across SANs.

                It makes less sense if you use ASM to mirror (on the same SAN) 2 RAID5 LUNs.

                You also may have no choice. For example, migrating a high redundancy diskgroup from a storage server (no redundancy by server), to a SAN (RAID5/10 redundancy by SAN). When migrating ASM diskgroups, you cannot reduce redundancy. You cannot make a high redundancy diskgroup a normal redundancy diskgroup. Or a normal one, an externally redundant diskgroup.

                In such a case you normal/high redundancy diskgroup on the new SAN (with its RAID5/10 LUNs), still need its own mirrors in ASM.
                3. Which type of rebalancing (from zero to 11) is the best to use, considering the overall performance , I mean, when you have to add a new disk, ASm will perform the rebalancing.
                How busy is the I/O layer? What production processing is running? How will this be impacted by using power 11? How soon does the old (dropped) disks need to be physically decommissioned? Etc.

                There is no single correct answer to what load balancing power setting to use.
                4. Which is better to use: ASM Cluster File System or Oracle Cluster File System ?
                For what purpose?

                For RAC database purposes only? ACFS. As OCFS adds another set of kernel drivers, kernel services, and another Interconnect (ocfs needs its own IP Interconnect that is typically run over the public IP network of the cluster, and that can be/should be run over the private IP network of the cluster).

                Why add more moving parts (ocfs) when ACFS is available?

                Then again - why use ACFS when you are not running Grid Infrastructure? Or why makes your cluster file system dependent on Grid/CRS to be running (with ASM), when OCFS's moving parts are significantly fewer and less complex?

                As with most things in Information Technology - there is no overall "best". The "best" depends on a number of factors, and the business requirements that need to be met.
                • 5. Re: ASM, rebalancing, 3rd party storage and performance
                  Dude!
                  ASM is a software based implementation to provide data redundancy, but it is not RAID and works on the basis of file extents (AU units) distributed on available free space of disk failure groups. ASM with only 2 disk failure groups is practically the same as RAID 1, but the situation changes drastically with multiple disk failure groups. ASM, unlike typical static RAID solutions provides more configuration and space management flexibility. For instance you can add disks without having to rebuild your RAID configuration from scratch. From a performance, redundancy and space utilization aspect, I bet that multiple ASM failure groups (single or multiple disks) can provide a better value for money than any RAID solution, in particular for Oracle Database products.