5 Replies Latest reply: Sep 16, 2011 9:12 AM by Marc Fielding RSS

    Exadata - Q related to Multipathing and IDP

    Lyxx
      Hi All -

      I have had some chances to attend few recorded webinar for Exadata, Since, Exadata documentation is not freely available, I have below doubts

      1 - Multipathing : I am not a storage guy and being a DBA i have been hearing some time we had performance issue because of trespassing. So we had to install multipathing software
      at DB Node to achieve multipath from DB node to Storage server for load balancing + failover.

      -- I agree here I have not 100% clear understanding about multipathing and got above expression.

      But I could not find any related material how Exadata addressed that, I could understand it has got 2 Infiniband connectivity but top of that I did not find if any software
      was required or it is already installed.

      - I would appreciate if someone can help me to understand multipathing with Exadata, how exactly it works and provides the same capability.

      2 - ASM IPD : works with JBOD, is this stands true with Exadata ? I am aware about cell disk partitioning which can be mapped to multiple grid disk ( hot / cold ) .. Just want to understand
      if these are 2 diff. concepts ( i think yes, ) but thought to clarify ..

      Regards,
        • 1. Re: Exadata - Q related to Multipathing and IDP
          Marc Fielding
          Hello Lyxx,

          1. Multipathing: since Exadata communicates with storage over a network interface it can use the same OS-based interface failover as non-Infiniband network interfaces. In the specific Exadata case, it uses the Linux bonding driver; look at the /etc/sysconfig/network-scripts/ifcfg-bond* files to see how it's configured.
          2. ASM IPD: I'm not quite sure what you mean by IPD. IPD/OS is an old name for what's now the cluster health monitor, but I don't see how this relates to disk partition layout.

          Marc
          • 2. Re: Exadata - Q related to Multipathing and IDP
            Lyxx
            Hello Marc

            Thank you very much for replying,

            1 - Multipathing : What are the pros & cons of connecting storage server with network interface ( IB over 40GBps ) as oppose to have Fiber channel connectivity? any thought on this ?


            2 - ASM IDP : I am so sorry for the confusion. I put IPD instead of IDP which is ASM intelligent Data Placement which works with JBOD. ASM leverage geometric info disks to place hot data on outer
            portion. My point was to distinguish between Cell disk partition ( using interleaving outer/inner circle ) Vs ASM IDP functionality of ASM which is not exadata dependent.


            Thanks again for your help.

            Regards,
            • 3. Re: Exadata - Q related to Multipathing and IDP
              Lyxx
              Hi All -

              can anyone please have a look to above post ?

              Regards,
              • 4. Re: Exadata - Q related to Multipathing and IDP
                gsalem-Oracle
                Lyxx wrote:
                Hello Marc

                Thank you very much for replying,

                1 - Multipathing : What are the pros & cons of connecting storage server with network interface ( IB over 40GBps ) as oppose to have Fiber channel connectivity? any thought on this ?
                It's faster, and in the case of Exadata, it is 'private': the storage, neither the network used to access it, is shared with other machines
                >
                >
                2 - ASM IDP : I am so sorry for the confusion. I put IPD instead of IDP which is ASM intelligent Data Placement which works with JBOD. ASM leverage geometric info disks to place hot data on outer
                portion. My point was to distinguish between Cell disk partition ( using interleaving outer/inner circle ) Vs ASM IDP functionality of ASM which is not exadata dependent.
                It more or less comes back to the same thing, in one way, you use the cells to do this, in the other you tell ASM to do it for you. In the later case, with exadata as with any other disk system, if you partition the disks before giving them to ASM, part of IDP is futile, as placing data intelligently on the inner partitions will not give you that much benefit.
                >
                Thanks again for your help.

                Regards,
                • 5. Re: Exadata - Q related to Multipathing and IDP
                  Marc Fielding
                  Hello Lyxx,

                  I've found the actual failover time of fiber channel and Exadata InfiniBand to be quite similar, assuming everything is configured properly. Which is one of the major benefits of Exadata, as it comes pre-configured. InfiniBand also has much more capacity without the complex and expensive HBA aggregation in FC. The main advantages of FC is that you can use a storage fabric for existing non-Exadata systems. FC is also mature and well-understood, and many companies already have FC expertise in-house while InfiniBand is tougher to find (though you have Oracle to help you in the case of Exadata).

                  Regarding intelligent data placement, this can actually be done in any ASM system with direct-attached disk; for years, I/O intensive Oracle shops have only formatted the outer tracks of their disks to get faster throughput and better spindles-to-capacity ratios. However you can't use IDP when there's a RAID controller striping data evenly across the disks, such as most SANs.

                  Cheers,

                  Marc