This discussion is archived
3 Replies Latest reply: Nov 2, 2012 9:35 AM by cindys RSS

ZFS Mirror resync?

user9315097 Newbie
Currently Being Moderated
Suppose I have a mirrored zpool and detach one half, and later reattach it. That will cause a resilver which will take many hours (or days) if the zpool is large even if only a small percent of the files have changed in the interim. I'm not sure how resilver works but I assume it at least ends up reading all the in use blocks in both pools.

Is there some way to detach/reattach so that zfs keeps track of what changed in the interim and only uses that info to resync?
  • 1. Re: ZFS Mirror resync?
    cindys Pro
    Currently Being Moderated
    True, that if you detach a disk from a ZFS storage pool and re-attach it, it will resilver the entire device.
    We are working on some resilvering performance improvements that will improve the overall resilvering
    process but the attach/detach operation will work similarly and will resilver the entire disk.

    I believe that offlining/onlining a pool device would just resilver the deltas. You could experiment on a test
    pool (and no important data) with this approach, offline a mirrored pool device, physically/logically disconnect it,
    reconnect it and online the device.

    I don't recommend this approach because it opens a window of vulnerability by offlining a device that might
    already be going bad or some other problem occurs simultaneously.

    A follow-up question is why are you detaching mirrors?

    I would endeavor to keep all my mirrored disks attached.

    Thanks, Cindy
  • 2. Re: ZFS Mirror resync?
    user9315097 Newbie
    Currently Being Moderated
    A follow-up question is why are you detaching mirrors?
    The thought was to do a 3 way mirror and detach the third one during the day and reattach/sync it at night. The idea was that what ever non-recoverable failure happened to the 2 way during the day, there would still be a way to roll back a day with the detached disk (until the next resync).

    This would have the same effect as using a 1 day snapshot, but would survive even if the online pool self destructed.

    We could also do this by using a snapshot and zfs send, but that is dependent on network throughput. Thinking out loud I wonder if we couldn't keep the third copy as a different unmirrored zpool and do snapshot/zfs send on the same server every night without using the network.

    At any rate it sounds like we should test offline/online not detach/attach.
  • 3. Re: ZFS Mirror resync?
    cindys Pro
    Currently Being Moderated
    Now that I know what you are doing and I almost asked about 3-way mirrors, then I would stick with
    zpool attach and detach and possibly, zpool split.

    If you are looking for best redundancy and replication, then I would do something like:

    1. Take daily snapshots
    2. Attach a 3rd disk to do a 3-way mirror at the beginning of the week
    3. Split the pool at end of the week as a mirrored 2-disk (original) pool and a 1 disk (backup) pool
    4. Backup the backup pool
    5. Repeat steps 2-4, attach, split, backup, attach

    If you need to mediate the resilver time, do the attach in the evening. Or, switch steps 2 and 3,
    split the pool at the beginning of the week and reattach the 3rd disk on Friday and resilver over
    the weekend (this makes more sense).

    Thanks, Cindy


  • Correct Answers - 10 points
  • Helpful Answers - 5 points