8 Replies Latest reply: Nov 9, 2012 11:08 AM by Cindys-Oracle RSS

    Help understanding disk partitions / slices

    972338
      Long story short(ish):

      Was running with rpool on a hardware mirror. One of the disks failed and we're unable to obtain a suitable replacement in a timely fashion. I attached another disk to the controller with the plan to add it to rpool, wait for resilvering, and then destroy the original hardware mirror and use the reclaimed disk in rpool, basically going from hardware mirroring to zfs mirroring.

      So, I ran format and added a solaris partition to the new disk. Then, I run:

      zpool attach rpool c2t0d0s0 c2t19d0s0

      and get:

      cannot open '/dev/dsk/c2t19d0s0': I/O error

      However, 'zpool attach rpool c2t0d0s0 c2t19d0p0' works fine.

      I understand that in order to boot from the new disk, I need to install grub. Grub installs fine if I use c2t19d0s0, but not c2t19d0p0.

      I guess I'm just trying to understand what I'm working with here, and whether I'll end up with a bootable system this way, as I intend to remove c2t0d0s0 from rpool and recreate it with a different underlying hardware configuration.

      Thanks
        • 1. Re: Help understanding disk partitions / slices
          972338
          I think I figured it out.. I manually created partition (or is it slice?) 0. Here's the table:

          partition> print
          Current partition table (unnamed):
          Total disk cylinders available: 30269 + 2 (reserved cylinders)

          Part Tag Flag Cylinders Size Blocks
          0 root wm 1 - 30268 231.86GB (30268/0/0) 486255420
          1 unassigned wm 0 0 (0/0/0) 0
          2 backup wu 0 - 30268 231.87GB (30269/0/0) 486271485
          3 unassigned wm 0 0 (0/0/0) 0
          4 unassigned wm 0 0 (0/0/0) 0
          5 unassigned wm 0 0 (0/0/0) 0
          6 unassigned wm 0 0 (0/0/0) 0
          7 unassigned wm 0 0 (0/0/0) 0
          8 boot wu 0 - 0 7.84MB (1/0/0) 16065
          9 unassigned wm 0 0 (0/0/0) 0


          I then was able to add it to rpool:

          root@hermes:~# zpool attach rpool c2t0d0s0 c2t19d0s0
          invalid vdev specification
          use '-f' to override the following errors:
          /dev/dsk/c2t19d0s0 overlaps with /dev/dsk/c2t19d0s2
          root@hermes:~# zpool attach -f rpool c2t0d0s0 c2t19d0s0
          Make sure to wait until resilver is done before rebooting.
          root@hermes:~# zpool status -v rpool
          pool: rpool
          state: ONLINE
          status: One or more devices is currently being resilvered. The pool will
          continue to function, possibly in a degraded state.
          action: Wait for the resilver to complete.
          scan: resilver in progress since Sat Nov 3 16:06:29 2012
          11.9M scanned out of 24.0G at 871K/s, 8h0m to go
          11.9M resilvered, 0.05% done
          config:

          NAME STATE READ WRITE CKSUM
          rpool ONLINE 0 0 0
          mirror-0 ONLINE 0 0 0
          c2t0d0s0 ONLINE 0 0 0
          c2t19d0s0 ONLINE 0 0 0 (resilvering)

          errors: No known data errors


          So, is the message "/dev/dsk/c2t19d0s0 overlaps with /dev/dsk/c2t19d0s2" normal? I looked at the partition table for the current rpool disk, and besides it being a different (smaller) size, the table was the same:

          partition> print
          Current partition table (original):
          Total disk cylinders available: 19327 + 2 (reserved cylinders)

          Part Tag Flag Cylinders Size Blocks
          0 root wm 1 - 19326 148.04GB (19326/0/0) 310472190
          1 unassigned wm 0 0 (0/0/0) 0
          2 backup wu 0 - 19326 148.05GB (19327/0/0) 310488255
          3 unassigned wm 0 0 (0/0/0) 0
          4 unassigned wm 0 0 (0/0/0) 0
          5 unassigned wm 0 0 (0/0/0) 0
          6 unassigned wm 0 0 (0/0/0) 0
          7 unassigned wm 0 0 (0/0/0) 0
          8 boot wu 0 - 0 7.84MB (1/0/0) 16065
          9 unassigned wm 0 0 (0/0/0) 0

          partition> quit


          What say the experts?
          • 2. Re: Help understanding disk partitions / slices
            Dave Miner-Oracle
            It's a longstanding convention in Solaris that slice 2 is defined to be the whole disk (or, in the case of x86, the whole Solaris partition), so it's normal to have overlaps between s2 and other slices. If you really wanted to use the entire space as part of the ZFS pool you could have just use s2 without defining s0 and achieved the same result.

            With 11.1 and later we recommend using GPT labeling instead, which gets rid of the two layers of partitions and slices.
            • 3. Re: Help understanding disk partitions / slices
              Cindys-Oracle
              The long-standing boot requirement for disk slice makes the disk replacement process more
              complicated for root pools and our available disk partitions/slices also complicate trying to
              just use a disk for a ZFS root pool.

              The p* devices should not be used in a ZFS storage pool.

              You can read about the root pool requirements here:

              http://docs.oracle.com/cd/E26502_01/html/E29007/zfsboot-2.html#gitgn

              ZFS Root Pool Configuration Requirements

              If this is an x86 system and c2t19d0s0 is a valid slice, this error is a bug:

              root@hermes:~# zpool attach rpool c2t0d0s0 c2t19d0s0
              invalid vdev specification
              use '-f' to override the following errors:
              /dev/dsk/c2t19d0s0 overlaps with /dev/dsk/c2t19d0s2

              Sorry about that.

              Thanks, Cindy
              • 4. Re: Help understanding disk partitions / slices
                972338
                Thanks for the replies. Unfortunately, I think my plan won't work.... I removed c2t0d0s0 from rpool and when I went to re-add it, I now get "cannot attach c2t0d0s0 to c2t19d0s0: device is too small"

                And that is true. c2t19d0s0 is larger. Any non-destructive remedy for this?
                • 5. Re: Help understanding disk partitions / slices
                  Cindys-Oracle
                  No easy way, unfortunately. If the attaching or replacing disk is larger than the
                  existing disk, then you're fine, but you can't go the other way yet.

                  Is c2t0d0 truly smaller or is just the slice 0 smaller? If so, repartitioning to
                  increase s0 would be best but other than that, I don't see a solution other
                  than finding another larger disk.

                  Thanks, Cindy
                  • 6. Re: Help understanding disk partitions / slices
                    972338
                    Ok, thanks. I have another drive of the same nominal size size that luckily is slightly larger. I need to verify that I can boot from from the curent rpool, but I think I have everything in order.. unfortunately the 2 drives dedicated to rpool are internal to the system and as such not hot-swappable. All the data is on hot swappable drives, thankfully...
                    • 7. Re: Help understanding disk partitions / slices
                      972338
                      Thanks for the help. Everything is back to optimal. Was able to boot from the replaced disk just fine, and added a second of the same size and attached it to rpool.

                      BTW, i tested a bit in a VM before doing this and noted that I didn't need to install grub on the second disk in order to boot from it after adding it to rpool and resilvering. I didn't see anything about this in the Oracle documentation but other sources via google suggested installing grub. I tried it both ways and found no difference.
                      • 8. Re: Help understanding disk partitions / slices
                        Cindys-Oracle
                        True. If you attach a new second disk to the root pool disk, then the
                        boot info is applied automatically. If you use zpool replace to replace
                        a root pool disk, then you still need to apply the boot blocks automatically.

                        The ZFS Admin describes the zpool attach behavior here, but its
                        not entirely clear that zpool replace does not so I will add it:

                        http://docs.oracle.com/cd/E26502_01/html/E29007/gjtuk.html#gjtui

                        How to Replace a Disk in a ZFS Root Pool (SPARC or x86/VTOC)

                        Although the embedded example includes the zpool replace and bootloader
                        steps.

                        Thanks, Cindy