1 2 Previous Next 17 Replies Latest reply on Jul 30, 2012 1:13 PM by Honza

    Local storage not available

    Honza
      Hello,

      I am trying to create a non-clustered server pool with a single VM server 3.1.1, using Manager also at 3.1.1.

      So far I've created a Server Pool and added the discovered server to this pool.
      The next step would be to create a repository on this server by going to Repositories tab, add a repository and click the "physical disk" radio button. At this point I am able to select the desired Server Pool.
      However, I don't see any available disks to choose as the repository - only blank combo boxes:
      http://d.pr/i/4eYY
      http://d.pr/i/T0G3
      This looks to me like a wrong window, since the label is for a san server and not a physical disk array or something along those lines.

      The internal server disks are arranged in a raid configuration and a single partition.

      I can't figure out what I'm missing. Please advise.
        • 1. Re: Local storage not available
          Honza
          Small update, the server doesn't show any physical disks either:
          http://d.pr/i/suQP

          So, how do I make sure physical disks get discovered along with the server itself? Or am I on the wrong path completely?

          Regards.
          • 2. Re: Local storage not available
            nasamang
            What make/model is your hardware, also what RAID controller?

            HP DL380/5's have known issues because Oracle uses an unsupported driver.
            1 person found this helpful
            • 3. Re: Local storage not available
              user12273962
              To see local storage to the VM the disk must be empty. You can not make one large partition on your raid array and use that same partition for your VM installation... and then expect to see the free space available to be used for a repo.

              Check out page 49 of the user guide

              As long as no partition and data are present the device will be detected as a raw disk. The choice is yours
              to use the local disks either to provision logical storage volumes as disks for virtual machines or to install a
              storage repository. If you place a storage repository on the local disk, an OCFS2 file system is installed.

              You will have to break up your RAID volume into multiple LUNS. You should then be able to see the unused LUNS. I've personally never done this but I do have some SSD drives I use for extending database cache. I can see these drives just fine locally. The key is there can't be a file system on the disks. Oracle VM expects a raw device.
              1 person found this helpful
              • 4. Re: Local storage not available
                Honza
                >
                What make/model is your hardware, also what RAID controller?

                HP DL380/5's have known issues because Oracle uses an unsupported driver.
                >
                Its a proliant ml150 g6 server, with a P410i Smart Array Controller.
                The controller is the same as mentioned in note 1373817.1, which is probably what you are looking at as well. I've experimented with that solution with no positive results yet.

                >
                To see local storage to the VM the disk must be empty. You can not make one large partition on your raid array and use that same partition for your VM installation... and then expect to see the free space available to be used for a repo.

                Check out page 49 of the user guide

                As long as no partition and data are present the device will be detected as a raw disk. The choice is yours
                to use the local disks either to provision logical storage volumes as disks for virtual machines or to install a
                storage repository. If you place a storage repository on the local disk, an OCFS2 file system is installed.

                You will have to break up your RAID volume into multiple LUNS. You should then be able to see the unused LUNS. I've personally never done this but I do have some SSD drives I use for extending database cache. I can see these drives just fine locally. The key is there can't be a file system on the disks. Oracle VM expects a raw device.
                >

                Oh, I shouldn't have said a single partition, as far as partitions go, it's the default layout: /, /boot, swap and free space (it's a single drive as far as RAID goes). Can that work?

                Thanks for that info, I might have to reconfigure the RAID as you suggest, I wasn't aware that it must be a raw device, that brings much valued insight :-) Although I should have expected this since VM 2.1 probably works in a similar way, I just never used the VM Manager and didn't study repos.py closely.
                • 5. Re: Local storage not available
                  user12273962
                  I shouldn't have said single partition too. :)

                  Regardless of # of partitions, VM stiill sees it as a single dev. Like /dev/sda/

                  If you do a fdisk -l from your VM server you will see that VM sees it as a single device with multiple partitions. The space must show up as a new device. The only way I know to do this is the break your RAID volume up into multiple LUNS. You just can't add a partition using the existing free space on the device. Again. I have never done this but I do have independent local disks that are connected to the same RAID controller on my VM servers that show as being available to create a repo.
                  1 person found this helpful
                  • 6. Re: Local storage not available
                    Honza
                    Ok I will definitely try breaking up the RAID array, since I've been messing around with different layouts and multipath blacklists for far too long to get to a suitable workaround :-D
                    I will report back, hopefully as soon as possible.
                    • 7. Re: Local storage not available
                      Honza
                      So I've created two logical volumes on the array along the lines of this:

                      HP Smart Array p410
                           SATA Array A
                                Logical Drive 1
                                Logical Drive 2

                      Then installed Oracle VM 3.1.1, specifying only the first drive to be used for installation, in the partition layout it simply shows two drives with new partitions on the first, second drive is left empty.

                      Here is what the OS sees after installation:
                      [root@kremilek ~]# fdisk -l
                      
                      Disk /dev/cciss/c0d0: 52.4 GB, 52428718080 bytes
                      255 heads, 63 sectors/track, 6374 cylinders
                      Units = cylinders of 16065 * 512 = 8225280 bytes
                      
                                 Device Boot      Start         End      Blocks   Id  System
                      /dev/cciss/c0d0p1   *           1          13      104391   83  Linux
                      /dev/cciss/c0d0p2              14         535     4192965   82  Linux swap / Solaris
                      /dev/cciss/c0d0p3             536        6374    46901767+  83  Linux
                      
                      Disk /dev/cciss/c0d1: 947.7 GB, 947742613504 bytes
                      255 heads, 32 sectors/track, 226845 cylinders
                      Units = cylinders of 8160 * 512 = 4177920 bytes
                      
                                 Device Boot      Start         End      Blocks   Id  System
                      [root@kremilek ~]# 
                      [root@kremilek ~]# df -h
                      Filesystem            Size  Used Avail Use% Mounted on
                      /dev/cciss/c0d0p3      44G  744M   41G   2% /
                      /dev/cciss/c0d0p1      99M   28M   67M  29% /boot
                      tmpfs                 314M     0  314M   0% /dev/shm
                      none                  314M   40K  314M   1% /var/lib/xenstored
                      [root@kremilek ~]# 
                      There are two drives that appear as separate devices.
                      Oracle VM Manager still doesn't recognize any physical disks.
                      Is this strange or just wrong procedure? :-D I will try to leave the second logical volume uninitialized, during installation, if possible.
                      • 8. Re: Local storage not available
                        Honza
                        If I leave the second logical volume uninitialized (no partition table) then the server cannot be added to a server pool, with an error complaining about an error component.
                        Why does this have to be so difficult, sure the new Manager interface finally seems usable, but using the server itself just seemed more straightforward with 2.1.1

                        Edit:
                        It can be coupled with a server pool if the pool is recreated adding the server in the process of pool creation.

                        However, all of this might have been for nothing, since looking back at the repository creation, the popup dialog still only offers SAN servers and not a disk array (screenshot above). It's different from what can be found in the user guide.

                        Edited by: Honza on Jul 27, 2012 4:23 AM
                        • 9. Re: Local storage not available
                          user12273962
                          I don't see the screenshot.

                          Are you trying to create a server pool and a repo on the same device?

                          Did you create an non clustered server pool?
                          1 person found this helpful
                          • 10. Re: Local storage not available
                            Honza
                            >
                            I don't see the screenshot.
                            >
                            It's the two links in my first post. The second image is the popup I get when clicking on the magnifying class icon.

                            >
                            Are you trying to create a server pool and a repo on the same device?

                            Did you create an non clustered server pool?
                            >

                            I have a server on which I want to use the internal disk array as a storage repository and to run all the VMs as well.
                            VM Manager is on a separate machine of course. Not sure what the direct connection is between a server pool and the physical server itself.
                            When creating the server pool, I have unchecked the clustered option.
                            • 11. Re: Local storage not available
                              user12273962
                              Okay here we go. I just reconfigured one of spare servers to do this and its not that difficult. I created two raid 1 LUNS on the same raid controller. On my server they show up as sda and sdb. Installed vm on sda and left sdb alone.

                              First create a non clustered pool.

                              Add your discovered VM server to that pool

                              Select the VM server from "Server VMS" tab and change the perspective to "Physical Disks". You should see the disk/LUN in the list. Go to the repo tab and click the green plus to add a repo. From that point you can select "physical disk". Select the non clusterd pool you created and then you can then hit the drop down to select the disk which be named something along the lines of "generic local storage @ the VM server in the selected pool"

                              The can then assign the VM server to the repo and you're done. I didn't import any templates or anything but I did create a 10GB non sparse virtual disk just see if it worked. It was fast. Worked well.

                              Hope this helps. I wanted to know how to do it anyway... just in case I ever choose to do it myself. I figured it was along those lines but I've haven't done till now. The disk will not show up under the "storage" tab under local file systems until you add the repo to it. It will then show up.

                              Edited by: user12273962 on Jul 27, 2012 7:54 AM
                              • 12. Re: Local storage not available
                                Honza
                                Thank you for trying that out!
                                I believe I have an almost identical setup as you described though (Two logical volumes showing up as two devices), there must be some small step different.

                                Probably the RAID configuration and what can be identified by the VM agent, some different configuration with my hardware RAID, could even be an unsupported controller, but I refuse to accept that just yet. The only minor difference I see is that my devices don't get identified as /dev/sda, but /dev/cciss/c0d0 and /dev/cciss/c0d1 - the naming convention shouldn't matter though, the sub-structure might.

                                Can you please provide the following details about your VM Server installation?
                                1. Did you initialize the second device? I get a message for both devices telling me they need to be initialized (I believe this creates a partitioning table), happens only after fresh LVM creation.
                                2. On one of the installation screens there is a checkbox for multipath access, did you have that checked?
                                3. You mention separate "LUNS", which although can be a general term for logical volumes, any chance you're using a SAN server? Could you please provide details for your RAID configuration? This might be a dead end, but I suspect the logical partitioning to be the culprit here.

                                In the mean time, I will try an installation directly on the physical disks, turning off the RAID altogether, just to troubleshoot.

                                Thanks!
                                • 13. Re: Local storage not available
                                  Honza
                                  Alright, lets close this thread.
                                  The configuration of using SAS internal disks is not supported on 3.1.1, I will have to use an older version, probably 3.0.3. Bummer.

                                  Here is a link to another discussion here on the forums.
                                  https://cn.forums.oracle.com/forums/thread.jspa?messageID=10473423
                                  • 14. Re: Local storage not available
                                    user12273962
                                    It worked fine for me and I'm using 10,000 rpm 146GB and 300 GB SAS drives with RAID1.

                                    1. I did not format it with a filesystem nor did I create a partition on the LUN.

                                    2. No. It is a local RAID1 LUN on a local LSI RAID controller.

                                    3. Not using a SAN disks.

                                    It might just be your hardware like you mentioned. I'm using Sun Servers with LSI raid controllers. The LSI raid controller doesn't have the ability to create varying Logical drives across the same RAID group. Thus, the Logical drive matches the total space available on the RAID1 group of disks.
                                    1 2 Previous Next