1 2 3 Previous Next 40 Replies Latest reply on Jan 10, 2013 12:43 PM by rukbat

    iSCSI Broken after 11.1 Update


      I am currently using Solaris 11 as an iSCSI target; however after updating to 11.1 and ZFS 34 My Linux clients stopped booting from the LUN's it seems they can read the MBR and bootloader but that is about all. I've tried adding some new zfs LUN's and the Windows and Linux Initiators can see them but give out I/O errors or freeze up when trying to Initialize or partition the new disks.

      Here are some of the things I've also tried;

      Removing all stmf and iscsi configuration e.g. all views,groups,LUN's and targets.
      Uninstalling/Re-install the storage-server package group
      Using a File as the LUN
      Adding a fresh disk with a brand new ZFS pool and using that as the LUN.
      Changing record sizes on LUN's e.g. 512K,4096K,8096K

      There is nothing in the log's to indicate a problem and I'm at my whits end as to what to do next.

      Anyone else having issues with this or can recommend a way to troubleshoot the problem.


        • 1. Re: iSCSI Broken after 11.1 Update
          I'm having the same issue. :-/

          I wish I would have searched for this a long time ago. I've easily burned 10 hours troubleshooting this today (I initially thought it had something to do with ZFS send/receive being broken).

          Edit -- this is on a clean install, btw. Nothing fancy. Just create a ZFS volume, create an LU, create a view, and then try to hit it with any initiator. You can read everything, but writes have bizarre behavior.

          Edited by: 969046 on Nov 1, 2012 9:51 PM
          • 2. Re: iSCSI Broken after 11.1 Update
            Ok thanks for the heads up, I was just about to do a full re-install. Hard to know what to do next as I don't have a support contract so even if they do release a patch it's not as if I can use it. OpenIndianna is an option but it sounds like this is also being abandoned. I still want to keep my NAS's Solaris based as I like the new ZFS features and COMSTAR.
            • 3. Re: iSCSI Broken after 11.1 Update
              same issue here

              removed the lun and recreated it and adding it to the view made it viewable but nothing can connect or do anything with it
              • 4. Re: iSCSI Broken after 11.1 Update
                Same problem here. Tried it with Linux, Windows and Solaris itself. Got it working on loopback device with very slow transfer speed. Is a bug report about this issue already filed by someone?

                Edited by: user3146817 on 04.11.2012 07:34

                Edited by: user3146817 on 04.11.2012 07:34
                • 5. Re: iSCSI Broken after 11.1 Update
                  i reinstalled 11.1 and imported the pool re-setup iscsi
                  vmware will see it and tries to import it but it wont appear as a datastore

                  cant roll back because zpool is v34
                  • 6. Re: iSCSI Broken after 11.1 Update
                    I was going to check for a bug report but it appears I can't even view support.oracle.com without a support contract. I did have an expired Solaris 10 contract which allowed me to at least view patches and bugs until recently, but now nothing. Aarggghhhh!
                    • 7. Re: iSCSI Broken after 11.1 Update
                      I'm going to try to reproduce this problem this week.

                      Please confirm that this is correct:

                      1. Create S11 iSCSI target
                      2. Create s11 iSCSI initiator
                      3. Share data between initiator and target
                      4. Upgrade both systems to S11.1

                      I'm slightly curious whether you can create a pool and a file system on the S11.1 server,
                      share the file system over NFS and see if it is accessible on the S11.1 client.


                      • 8. Re: iSCSI Broken after 11.1 Update
                        Hi Cindy,

                        I haven't tried Solaris Initiator to Solaris Target but maybe it works. I've only tried Linux/Windows7 Initiators to the Solaris Target. Also NFS/CIFS creating zpools and slices all work fine in S11.1 for me.

                        • 9. Re: iSCSI Broken after 11.1 Update
                          these are my 2 configuration

                          recreate zpool
                          create volume zfsr10/iscsi
                          create volume zfsr6/iscsi
                          sbdadm create-lu zfsr10/iscsi
                          sbdadm create-lu zfsr6/iscsi
                          stmfadm add-view <GUID>

                          vmware connects via iscsi and adds the zfsr10 lun and formats it as vmfs5 then create some machines on this lun
                          windows 2008 r2 connects via iscsi and adds the zfsr6 lun as GPT and formats it as NTFS then some files and folders are added to it.

                          shutdown all machines in vmware on zfsr10 to make sure nothing writes to the disk, shutdown the windows 2008 machine so it doesn't write to the zfsr6 lun.

                          upgrade solaris from 11 to 11.1

                          reboot solaris

                          rescan disk in vmware - disk does not appear > goto add new lun, which shows up but asks for reformat
                          removed the sbdadm delete-lu and recreate a new lu then add-view to the new guid
                          vmware sees the lun when add it, instead of asking to format it asks if you want to keep existing signature > yes
                          add process is complete but lun does not appear

                          windows 2008 will take a long time to load before it cant read the lun, it will mark the disk as offline. If you bring it online it will ask to be formatted.
                          • 10. Re: iSCSI Broken after 11.1 Update
                            I have the same problem.
                            Here is my procedure to produce the problem:

                            new install solaris 11.1
                            create zvol
                            create lu, view, target.
                            windows or vmware esxi can see the target, but cannot do anthing on it.
                            windows shows I/O error. esxi shows error when create datastore.

                            this procedure on solaris 11/11 works fine.

                            need someone to fire a bug report. 3Q ;-)

                            Edited by: user6951419 on Nov 5, 2012 9:47 PM
                            • 11. Re: iSCSI Broken after 11.1 Update
                              I'm trying to get some more help to troubleshoot your problems from the team that would know
                              because your configs are beyond my skills.

                              If you could provide some error messages that would be helpful.

                              I could not reproduce this by configuring the basic COMSTAR target/iSCSI initiator configuration.

                              See below.


                              1. On the S11 FCS target system, do the following:

                              # pkg install group/feature/storage-server
                              # svcadm enable stmf
                              # zpool create sanpool mirror c2t3d0 c2t4d0
                              # zfs create -V 2g sanpool/vol1
                              # stmfadm create-lu /dev/zvol/rdsk/sanpool/vol1
                              # stmfadm add-view 600144F03B3A810000005098D2380001
                              # itadm create-target
                              # itadm list-target -v


                              2. On the S11.1 FCS initiator system, do the following:

                              # svcadm enable network/iscsi/initiator
                              # iscsiadm add static-config guid,IP-addr
                              # iscsiadm list static-config
                              # iscsiadm modify discovery --static enable


                              The LUN is available in the format output.

                              Create a ZFS storage pool:
                              # zpool create tank c0t600144F03B3A810000005098D2380001d0
                              # zfs create tank/cindy
                              # /usr/dict/words /tank/cindy/file.1
                              # /usr/dict/words /tank/cindy/file.2

                              3. Update S11 FCS target system to S11.1 FCS.

                              The initiator wasn't happy after the reboot but that is expected:

                              NOTICE: iscsi connection(5) unable to connect to target guid, target address IP-addr

                              4. Post-S11.1 update, Create another vol2 on the target and make it available:

                              # zfs create -V 2g sanpool/vol2
                              # stmfadm create-lu /dev/zvol/rdsk/sanpool/vol2
                              Logical unit created: 600144F03B3A810000005098FAD80001
                              # stmfadm list-lu
                              LU Name: 600144F03B3A810000005098D2380001
                              LU Name: 600144F03B3A810000005098FAD80001
                              # stmfadm add-view 600144F03B3A810000005098FAD80001
                              # stmfadm list-view -l 600144F03B3A810000005098FAD80001
                              View Entry: 0
                              Host group : All
                              Target group : All
                              LUN : 1
                              # itadm create-target
                              # itadm list-target -v

                              On the initiator, make it available:

                              # iscsiadm add static-config guid,IP-addr

                              Now, both devices are available in format:

                              AVAILABLE DISK SELECTIONS:
                              0. c0t600144F03B3A810000005098D2380001d0 <SUN-COMSTAR-1.0-2.00GB>
                              1. c0t600144F03B3A810000005098FAD80001d0 <SUN-COMSTAR-1.0 cyl 1022 alt 2 hd 128 sec 32>

                              Create another pool:

                              # zpool create pond c0t600144F03B3A810000005098FAD80001d0
                              # zfs create pond/amy
                              # cp /usr/dict/words /pond/amy/file.1
                              • 12. Re: iSCSI Broken after 11.1 Update
                                Okay, some better news...

                                We have a description of this problem internally and they are doing some more testing
                                to better understand it. If you can wait until tomorrow we might be able to identify a
                                reasonable workaround that most likely be applied to a Linux or Windows initiator.

                                The Solaris 11.1 initiator/target configuration does not seem to be impacted.


                                • 13. Re: iSCSI Broken after 11.1 Update
                                  Thanks Cindy,

                                  Yeah Solaris Initiator to Solaris Target is not a problem for me either, in fact I was able to remote install the operating system onto the target. It's all other vendors to Solaris Target that is not working (which is the most common scenario in the real world). Hopefully they do the sensible thing and provide a patch to community users or re-release the 11.1 installer.

                                  • 14. Re: iSCSI Broken after 11.1 Update
                                    Asking users to fix it in the initiator seems like the wrong approach since that would require "fixes" to every potential Windows, Linux, Unix, VMware, Bios, etc, initiator out in the wild. Seems especially troublesome since all of these were working with the Solaris target software up to 11.1. Doesn't seem too likely that the rest of the world "broke" the day Solaris 11.1 was release...just sayin.
                                    1 2 3 Previous Next