This discussion is archived
7 Replies Latest reply: Dec 10, 2012 5:51 AM by User477708-OC RSS

Do ASM Rebalance Operations Run in Serial or Parallel?

976474 Newbie
Currently Being Moderated
Background: We are migrating a lot of databases from one SAN appliance to another. We are doing this by adding new disks from the new SAN appliance to the existing diskgroups, rebalancing, removing the old disks from the diskgroups, and then rebalancing again.

Question: If I execute two ALTER commands with the same power on 2 or more separate diskgroups, do both operations start executing right away? Or do they queue up and execute one after another?

I ask because we would like to queue up several re-balances so we don't have DBAs watching status bars all day.
  • 1. Re: Do ASM Rebalance Operations Run in Serial or Parallel?
    898553 Newbie
    Currently Being Moderated
    They'll do it in parallel though depending on the size of your diskgroups, a rebal with a power of 2 will take a while regardless.
  • 2. Re: Do ASM Rebalance Operations Run in Serial or Parallel?
    vlethakula Expert
    Currently Being Moderated
    ASM can perform only one rebalance operation at a time on a given instance

    From oracle documentation:

    http://docs.oracle.com/cd/B28359_01/server.111/b31107/asmdiskgrps.htm#BABCCEAB

    ASM can perform one disk group rebalance at a time on a given instance. Therefore, if you have initiated multiple rebalances on different disk groups, then Oracle processes this operation serially. However, you can initiate rebalances on different disk groups on different nodes in parallel.

    And also refer

    http://oraclehandson.wordpress.com/2010/03/25/asm-rebalancing-multiple-diskgroups-at-same-time/
  • 3. Re: Do ASM Rebalance Operations Run in Serial or Parallel?
    User477708-OC Journeyer
    Currently Being Moderated
    To speed up, you can also issue the add and drop in a single command so only 1 balance will occur. having had to do this recently, I created a proof of concept for this.

    Obviously milage may vary but for me, testing migrating 250gb from many smaller 64gb SATA luns to 1 large 15k disk on new san took 80 minutes. Querying the asm_operation I could see the storage move part of it estimated to take 45 minutes which after 45 minutes it sat at 0 minutes left for another 35 minutes. Im guessing the rebalance completed and it was doing statistics for those 35. The drop of the old disks took 40 minutes so 120 minutes in total.

    so I recreated the test and ran the add and drop in 1 command and it completed in 83 minutes. I used power 11. Heres an example... (not my blog btw)


    http://jhdba.wordpress.com/2012/05/29/asm-adding-and-dropping-disks-in-one-command/

    Im on 11.2.0.2, just check the version, not sure its available in earlier rounds.
  • 4. Re: Do ASM Rebalance Operations Run in Serial or Parallel?
    rcc50886 Journeyer
    Currently Being Moderated
    We are doing this by adding new disks from the new SAN appliance to the existing diskgroups, rebalancing, removing the old disks from the diskgroups, and then rebalancing again.
    This is not a optimal way to migrate the data. You don't need to rebalance while adding new disks. You simple doing the extra step that totally unnecessary.

    1. just add the new disks to diskgroup ---> no rebalance
    2. Drop disks with high rebalance level dependent upon how busy your system will be. Droping disks will allways do rebalance since the data needs to be distributed to remaining disks.
  • 5. Re: Do ASM Rebalance Operations Run in Serial or Parallel?
    User477708-OC Journeyer
    Currently Being Moderated
    rcc50886 wrote:
    We are doing this by adding new disks from the new SAN appliance to the existing diskgroups, rebalancing, removing the old disks from the diskgroups, and then rebalancing again.
    This is not a optimal way to migrate the data. You don't need to rebalance while adding new disks. You simple doing the extra step that totally unnecessary.

    1. just add the new disks to diskgroup ---> no rebalance
    2. Drop disks with high rebalance level dependent upon how busy your system will be. Droping disks will allways do rebalance since the data needs to be distributed to remaining disks.
    how do you add the disks with out rebalance happening and why would you? If you add the disks without specifying the REBALANCE and POWER parameter then Oracle uses the default database power rebalance setting of 1. You can query the v$asm_operation after you add the disks to see this.
  • 6. Re: Do ASM Rebalance Operations Run in Serial or Parallel?
    898553 Newbie
    Currently Being Moderated
    961469 wrote:
    rcc50886 wrote:
    We are doing this by adding new disks from the new SAN appliance to the existing diskgroups, rebalancing, removing the old disks from the diskgroups, and then rebalancing again.
    This is not a optimal way to migrate the data. You don't need to rebalance while adding new disks. You simple doing the extra step that totally unnecessary.

    1. just add the new disks to diskgroup ---> no rebalance
    2. Drop disks with high rebalance level dependent upon how busy your system will be. Droping disks will allways do rebalance since the data needs to be distributed to remaining disks.
    how do you add the disks with out rebalance happening and why would you? If you add the disks without specifying the REBALANCE and POWER parameter then Oracle uses the default database power rebalance setting of 1. You can query the v$asm_operation after you add the disks to see this.
    Say you want to add disks but wanted to hold off on any added load to the I/O subsystem then just do the rebal later. You can generate a XML file and import it using asmcmd chdg <xmlfile>. As long as the rebal power is set to "0", you're good to go.
  • 7. Re: Do ASM Rebalance Operations Run in Serial or Parallel?
    User477708-OC Journeyer
    Currently Being Moderated
    thanks for that reply. interesting, wouldnt have thought of that method. I rebalance frequently as I add disk as I need it. Whilst my storage guys manage the subsytem and monitor the load we never see so much pressure on a single disk rebalance were Id have to worry about it, even so while I still only ever rebalance outside of business hours I can see why you would do that "no rebalance" method inside of business hours.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points