3 Replies Latest reply: Nov 2, 2012 11:35 AM by Cindys-Oracle RSS

    ZFS Mirror resync?

      Suppose I have a mirrored zpool and detach one half, and later reattach it. That will cause a resilver which will take many hours (or days) if the zpool is large even if only a small percent of the files have changed in the interim. I'm not sure how resilver works but I assume it at least ends up reading all the in use blocks in both pools.

      Is there some way to detach/reattach so that zfs keeps track of what changed in the interim and only uses that info to resync?
        • 1. Re: ZFS Mirror resync?
          True, that if you detach a disk from a ZFS storage pool and re-attach it, it will resilver the entire device.
          We are working on some resilvering performance improvements that will improve the overall resilvering
          process but the attach/detach operation will work similarly and will resilver the entire disk.

          I believe that offlining/onlining a pool device would just resilver the deltas. You could experiment on a test
          pool (and no important data) with this approach, offline a mirrored pool device, physically/logically disconnect it,
          reconnect it and online the device.

          I don't recommend this approach because it opens a window of vulnerability by offlining a device that might
          already be going bad or some other problem occurs simultaneously.

          A follow-up question is why are you detaching mirrors?

          I would endeavor to keep all my mirrored disks attached.

          Thanks, Cindy
          • 2. Re: ZFS Mirror resync?
            A follow-up question is why are you detaching mirrors?
            The thought was to do a 3 way mirror and detach the third one during the day and reattach/sync it at night. The idea was that what ever non-recoverable failure happened to the 2 way during the day, there would still be a way to roll back a day with the detached disk (until the next resync).

            This would have the same effect as using a 1 day snapshot, but would survive even if the online pool self destructed.

            We could also do this by using a snapshot and zfs send, but that is dependent on network throughput. Thinking out loud I wonder if we couldn't keep the third copy as a different unmirrored zpool and do snapshot/zfs send on the same server every night without using the network.

            At any rate it sounds like we should test offline/online not detach/attach.
            • 3. Re: ZFS Mirror resync?
              Now that I know what you are doing and I almost asked about 3-way mirrors, then I would stick with
              zpool attach and detach and possibly, zpool split.

              If you are looking for best redundancy and replication, then I would do something like:

              1. Take daily snapshots
              2. Attach a 3rd disk to do a 3-way mirror at the beginning of the week
              3. Split the pool at end of the week as a mirrored 2-disk (original) pool and a 1 disk (backup) pool
              4. Backup the backup pool
              5. Repeat steps 2-4, attach, split, backup, attach

              If you need to mediate the resilver time, do the attach in the evening. Or, switch steps 2 and 3,
              split the pool at the beginning of the week and reattach the 3rd disk on Friday and resilver over
              the weekend (this makes more sense).


              Thanks, Cindy