This discussion is archived
9 Replies Latest reply: Feb 12, 2013 11:25 AM by 960874 RSS

Oracle VM Manager 3.1.1: Discovering SAN Servers

960874 Newbie
Currently Being Moderated
Even though I've gone through the discovery process for the iSCSI target iqn.201209.com.example.sn2:target1 (192.168.1.2) using the Oracle VM Manger 3.1.1 interface and both OracleVM 3.1.1 servers OVM-1 & OVM-2 have active sessions...
[root@ovm-1 ~]# iscsiadm -m node
192.168.1.1:3260,1 iqn.2011-09.com.example.sn1:target1
192.168.1.2:3260,1 iqn.2011-09.com.example.sn2:target1

[root@ovm-1 ~]# ls -1 /dev/disk/by-path/ip-192.168.1.2*
/dev/disk/by-path/ip-192.168.1.2:3260-iscsi-iqn.2011-09.com.example.sn2:target1-lun-1
/dev/disk/by-path/ip-192.168.1.2:3260-iscsi-iqn.2011-09.com.example.sn2:target1-lun-2
[root@ovm-2 ~]# iscsiadm -m node
192.168.1.2:3260,1 iqn.2011-09.com.example.sn2:target1
192.168.1.1:3260,1 iqn.2011-09.com.example.sn1:target1

[root@ovm-2 ~]# ls -1 /dev/disk/by-path/ip-192.168.1.2*
/dev/disk/by-path/ip-192.168.1.2:3260-iscsi-iqn.2011-09.com.example.sn2:target1-lun-1
/dev/disk/by-path/ip-192.168.1.2:3260-iscsi-iqn.2011-09.com.example.sn2:target1-lun-2
...the LUN's still are NOT displayed in the Oracle VM Manager 3.1.1 interface.

Ideas? Suggestions?

Eric Pretorious
Truckee, CA
  • 1. Re: Oracle VM Manager 3.1.1: Discovering SAN Servers
    budachst Journeyer
    Currently Being Moderated
    Do the targets show up under /dev/mapper ?
  • 2. Re: Oracle VM Manager 3.1.1: Discovering SAN Servers
    jurajl Newbie
    Currently Being Moderated
    Some suggestions:
    -make sure you have iscssi session OK: iscsiadm -m session
    -check if multipath is working fine: multipath -ll
    -make sure system time is correct on both nodes. and namager.
    -standard steps in Manager after OVM reinstall (ie.: perhaps rediscovering servers will help in your case):
    1. Discover Server
    2. Register Storage Array
    3. Present Repository
    4. Refresh Repository
    5. Rediscover Server

    :j
  • 3. Re: Oracle VM Manager 3.1.1: Discovering SAN Servers
    960874 Newbie
    Currently Being Moderated
    budachst wrote:
    Do the targets show up under /dev/mapper ?
    The targets do appear in the /dev/mapper directory of the two Oracle VM Server hosts +(but not the Oracle VM Manager host)+ but the directory contents are not the same (even though the iscsid sessions are the same):
    [root@ovm-1 ~]# ll /dev/mapper
    total 0
    brw-rw---- 1 root disk 252,   1 Oct  5 23:44 1IET_00010001
    brw-rw---- 1 root disk 252,   4 Oct  5 23:44 1IET_00010001p1
    brw-rw---- 1 root disk 252,   5 Oct  5 23:44 1IET_00010001p2
    brw-rw---- 1 root disk 252,   6 Oct  5 23:44 1IET_00010001p3
    brw-rw---- 1 root disk 252,   7 Oct  5 23:44 1IET_00010001p4
    brw-rw---- 1 root disk 252,   8 Oct  5 23:44 1IET_00010001p5
    brw-rw---- 1 root disk 252,   9 Oct  5 23:44 1IET_00010001p6
    brw-rw---- 1 root disk 252,  10 Oct  5 23:44 1IET_00010001p7
    brw-rw---- 1 root disk 252,   0 Oct  5 23:44 1IET_00010002
    brw-rw---- 1 root disk 252,   3 Oct  5 23:44 1IET_00010003
    brw-rw---- 1 root disk 252,   2 Oct  5 23:44 1IET_00010004
    crw------- 1 root root  10, 236 Oct  5 10:58 control
    brw-rw---- 1 root disk 252,  12 Oct  5 10:58 ovspoolfs
    [root@ovm-2 ~]# ll /dev/mapper
    total 0
    brw-rw---- 1 root disk 252,   0 Oct  5 23:45 1IET_00010001
    brw-rw---- 1 root disk 252,   2 Oct  5 23:45 1IET_00010002
    brw-rw---- 1 root disk 252,   3 Oct  5 23:45 1IET_00010003
    brw-rw---- 1 root disk 252,   1 Oct  5 23:45 1IET_00010004
    crw------- 1 root root  10, 236 Oct  2 21:03 control
    brw-rw---- 1 root disk 252,   5 Oct  3 18:26 dmnfs0
    brw-rw---- 1 root disk 252,   6 Oct  5 19:06 dmnfs1
    brw-rw---- 1 root disk 252,   4 Oct  2 21:03 ovspoolfs
    Eric Pretorious
    Truckee, CA
  • 4. Re: Oracle VM Manager 3.1.1: Discovering SAN Servers
    960874 Newbie
    Currently Being Moderated
    jurajl wrote:
    Some suggestions:
    -make sure you have iscssi session OK: iscsiadm -m session
    -check if multipath is working fine: multipath -ll
    -make sure system time is correct on both nodes. and namager.
    -standard steps in Manager after OVM reinstall (ie.: perhaps rediscovering servers will help in your case):
    1. Discover Server
    2. Register Storage Array
    3. Present Repository
    4. Refresh Repository
    5. Rediscover Server
    <ol>
    <li>Yes, iscsid is running on both Oracle VM Servers...</li>
    [root@ovm-1 ~]# iscsiadm -m session -P 3
    iSCSI Transport Class version 2.0-870
    version 2.0-872
    Target: iqn.2011-09.com.example.sn1:target1
       <<<snip>>>
              ************************
              Attached SCSI devices:
              ************************
              Host Number: 12     State: running
              scsi12 Channel 00 Id 0 Lun: 0
              scsi12 Channel 00 Id 0 Lun: 1
              scsi12 Channel 00 Id 0 Lun: 2
              scsi12 Channel 00 Id 0 Lun: 3
              scsi12 Channel 00 Id 0 Lun: 4
    Target: iqn.2011-09.com.example.sn2:target1
         Current Portal: 192.168.1.2:3260,1
         Persistent Portal: 192.168.1.2:3260,1
       <<<snip>>>
              ************************
              Attached SCSI devices:
              ************************
              Host Number: 13     State: running
              scsi13 Channel 00 Id 0 Lun: 0
              scsi13 Channel 00 Id 0 Lun: 1
              scsi13 Channel 00 Id 0 Lun: 2
    [root@ovm-2 ~]# iscsiadm -m session -P 3
    iSCSI Transport Class version 2.0-870
    version 2.0-872
    Target: iqn.2011-09.com.example.sn1:target1
       <<<snip>>>
              ************************
              Attached SCSI devices:
              ************************
              Host Number: 15     State: running
              scsi15 Channel 00 Id 0 Lun: 0
              scsi15 Channel 00 Id 0 Lun: 1
              scsi15 Channel 00 Id 0 Lun: 2
              scsi15 Channel 00 Id 0 Lun: 3
              scsi15 Channel 00 Id 0 Lun: 4
    Target: iqn.2011-09.com.example.sn2:target1
       <<<snip>>>
              ************************
              Attached SCSI devices:
              ************************
              Host Number: 14     State: running
              scsi14 Channel 00 Id 0 Lun: 0
              scsi14 Channel 00 Id 0 Lun: 1
              scsi14 Channel 00 Id 0 Lun: 2
    <li>Yes, multipath is running on both Oracle VM Servers...</li>
    [root@ovm-1 ~]# multipath -ll
    1IET_00010001 dm-1 IET,VIRTUAL-DISK
    size=25G features='0' hwhandler='0' wp=rw
    |-+- policy='round-robin 0' prio=1 status=active
    | `- 12:0:0:1 sdb 8:16 active ready running
    `-+- policy='round-robin 0' prio=1 status=enabled
      `- 13:0:0:1 sdd 8:48 active ready running
    1IET_00010004 dm-2 IET,VIRTUAL-DISK
    size=25G features='0' hwhandler='0' wp=rw
    `-+- policy='round-robin 0' prio=1 status=active
      `- 12:0:0:4 sdg 8:96 active ready running
    1IET_00010003 dm-3 IET,VIRTUAL-DISK
    size=25G features='0' hwhandler='0' wp=rw
    `-+- policy='round-robin 0' prio=1 status=active
      `- 12:0:0:3 sdf 8:80 active ready running
    1IET_00010002 dm-0 IET,VIRTUAL-DISK
    size=25G features='0' hwhandler='0' wp=rw
    |-+- policy='round-robin 0' prio=1 status=active
    | `- 12:0:0:2 sdc 8:32 active ready running
    `-+- policy='round-robin 0' prio=1 status=enabled
      `- 13:0:0:2 sde 8:64 active ready running
    [root@ovm-2 ~]# multipath -ll
    1IET_00010001 dm-0 IET,VIRTUAL-DISK
    size=25G features='0' hwhandler='0' wp=rw
    |-+- policy='round-robin 0' prio=1 status=active
    | `- 14:0:0:1 sdb 8:16 active ready running
    `-+- policy='round-robin 0' prio=1 status=enabled
      `- 15:0:0:1 sdd 8:48 active ready running
    1IET_00010004 dm-1 IET,VIRTUAL-DISK
    size=25G features='0' hwhandler='0' wp=rw
    `-+- policy='round-robin 0' prio=1 status=active
      `- 15:0:0:4 sdg 8:96 active ready running
    1IET_00010003 dm-3 IET,VIRTUAL-DISK
    size=25G features='0' hwhandler='0' wp=rw
    `-+- policy='round-robin 0' prio=1 status=active
      `- 15:0:0:3 sdf 8:80 active ready running
    1IET_00010002 dm-2 IET,VIRTUAL-DISK
    size=25G features='0' hwhandler='0' wp=rw
    |-+- policy='round-robin 0' prio=1 status=active
    | `- 15:0:0:2 sde 8:64 active ready running
    `-+- policy='round-robin 0' prio=1 status=enabled
      `- 14:0:0:2 sdc 8:32 active ready running
    <li>There's a *77-second difference* between the two Oracle VM Servers (i.e., the two iSCSI initiators):</li>
    [root@ovm-1 ~]# date
    Mon Oct  8 15:52:28 PDT 2012
    [root@ovm-2 ~]# date
    Mon Oct  8 15:53:45 PDT 2012
    <li>Forgetting+rediscovering may be an option, but I think that I'll reserve it for later. ;)</li>
    </ol>
    I think that I'll start by improving time-synchronization between the two Oracle VM Server nodes and then see about stoping/restarting the Oracle VM Manager node.
  • 5. Re: Oracle VM Manager 3.1.1: Discovering SAN Servers
    budachst Journeyer
    Currently Being Moderated
    This look all good to me. I can't imagine, why OVMM wouldn't discover the targets as valid local storage. Besides that synchronizing your VM server's clocks to OVMM (which is what I did), I don't really expect this to help in this case. I'd re-discover both VM servers and see, if they LUNs show un as local storage in their respective Physical Disks perspective.
    Maybe it would be a good idea to watch the ovm-agent.log on the VM servers, while re-discovering them.
  • 6. Re: Oracle VM Manager 3.1.1: Discovering SAN Servers
    jurajl Newbie
    Currently Being Moderated
    The time syncro is important - suggestion to syncronise it towards OVMM server is right - BTW this should be the default behaviour of OVM - to syncronize dom0 servers against OVMM server (check ntpq -p; default ntpd settings on your dom0 servers and make sure OVMM server has a good time ( or time source), aswell).
    BTW - the whole virtualised infrastructure should use ntp (even domU servers). See for inspiration "http://blogs.oracle.com/virtualization/entry/configuring_ntp_services_for_oracle"
    Taking a look into /var/log/ovs-agent.log on dom0 servers is another good suggestion.
    You should not need to unregister and register servers, start just by rediscovering them via OVMM gui (you can rediscover dom0, refresh Storage, refresh Repositories with no problem - even in infrastructure with running domU servers).
    :j
  • 7. Re: Oracle VM Manager 3.1.1: Discovering SAN Servers
    960874 Newbie
    Currently Being Moderated
    epretorious wrote:
    <ol start=4>
    <li>Forgetting+rediscovering may be an option, but I think that I'll reserve it for later. ;)</li>
    </ol>
    I think that I'll start by improving time-synchronization between the two Oracle VM Server nodes and then see about stoping/restarting the Oracle VM Manager node.
    After shutting down the Oracle VM Manager node and the two Oracle VM Server nodes for a day and restarting all three nodes, the two VM's that I'd created would not start. So I took the opportunity to delete both storage servers (iqn.2011-09.com.example.sn1:target1 & iqn.2011-09.com.example.sn2:target1) and start all over.

    +In the examples below, target of iqn.2011-09.com.example.sn1 has been renamed target2 as part of the troubleshooting process. This had no effect on OVM's behavior so I didn't bother changing it back.+

    What I found was quite interesting: Oracle VM Manager will not allow two LUN's with the same SCSI ID & LUN number! e.g., OVM will not use LUN's 1 & 2 of iqn.2011-09.com.example.sn1:target2 if iqn.2011-09.com.example.sn2:target1also has two LUN's numbered 1 & 2: e.g., this scenario does NOT work...
    [root@ovm-2 ~]# sudo iscsiadm -m session -P 3
    iSCSI Transport Class version 2.0-870
    version 2.0-872
    Target: iqn.2011-09.com.example.sn1:target2
         Current Portal: 192.168.1.1:3260,1
         Persistent Portal: 192.168.1.1:3260,1
         <<<snip>>>
              ************************
              Attached SCSI devices:
              ************************
              Host Number: 15     State: running
              scsi15 Channel 00 Id 0 Lun: 0
              scsi15 Channel 00 Id 0 Lun: 1
              scsi15 Channel 00 Id 0 Lun: 2
              scsi15 Channel 00 Id 0 Lun: 3
              scsi15 Channel 00 Id 0 Lun: 4
    Target: iqn.2011-09.com.example.sn2:target1
         Current Portal: 192.168.1.2:3260,1
         Persistent Portal: 192.168.1.2:3260,1
         <<<snip>>>
              ************************
              Attached SCSI devices:
              ************************
              Host Number: 10     State: running
              scsi10 Channel 00 Id 0 Lun: 0
              scsi10 Channel 00 Id 0 Lun: 1
              scsi10 Channel 00 Id 0 Lun: 2
    However, if I manually specify the SCSI ID or LUN ID of the four target LUN's of iqn.2011-09.com.example.sn1:target2, OVM will make all four available for use: e.g., this scenario works as expected...
    [root@ovm-1 ~]# sudo iscsiadm -m session -P 3
    iSCSI Transport Class version 2.0-870
    version 2.0-872
    Target: iqn.2011-09.com.example.sn1:target2
         Current Portal: 192.168.1.1:3260,1
         Persistent Portal: 192.168.1.1:3260,1
         <<<snip>>>
              ************************
              Attached SCSI devices:
              ************************
              Host Number: 17     State: running
              scsi17 Channel 00 Id 0 Lun: 0
              scsi17 Channel 00 Id 0 Lun: 5
              scsi17 Channel 00 Id 0 Lun: 6
              scsi17 Channel 00 Id 0 Lun: 7
              scsi17 Channel 00 Id 0 Lun: 8
    Target: iqn.2011-09.com.example.sn2:target1
         Current Portal: 192.168.1.2:3260,1
         Persistent Portal: 192.168.1.2:3260,1
         <<<snip>>>
              ************************
              Attached SCSI devices:
              ************************
              Host Number: 10     State: running
              scsi10 Channel 00 Id 0 Lun: 0
              scsi10 Channel 00 Id 0 Lun: 1
              scsi10 Channel 00 Id 0 Lun: 2
  • 8. Re: Oracle VM Manager 3.1.1: Discovering SAN Servers
    Looney128 Newbie
    Currently Being Moderated
    I had the same problems creating iSCSI connections to my SANs. I use the HP P4300 series and the way it manages the groups and clusters made it impossible to use with OVM 3.1.1.

    But good news. The new Beta 3.2.1 (build 358) seems to have fixed this problem.

    Edited by: Looney128 on Nov 13, 2012 8:31 AM
  • 9. Re: Oracle VM Manager 3.1.1: Discovering SAN Servers
    960874 Newbie
    Currently Being Moderated
    Looney128 wrote:
    I had the same problems creating iSCSI connections to my SANs. I use the HP P4300 series and the way it manages the groups and clusters made it impossible to use with OVM 3.1.1.

    But good news. The new Beta 3.2.1 (build 358) seems to have fixed this problem.
    Thanks, Looney!

    If OVS 3.2.1 is able to meet Oracle VM vs SuSE Linux Enterprise Server (SLES) I'll give OVMM 3.2.1 a try - otherwise it's just too damned much work to reprovision the servers that we're currently using!

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points