ZFS - Unable to offline drive in raidz1 based pool
Kurt Touet
ktouet at gmail.com
Tue Sep 22 17:07:24 UTC 2009
On Tue, Sep 22, 2009 at 6:56 AM, Pawel Jakub Dawidek <pjd at freebsd.org> wrote:
> On Sun, Sep 20, 2009 at 04:00:10PM -0600, Kurt Touet wrote:
>> I am using ZFS pool based on a 4-drive raidz1 setup for storage. I
>> believe that one of the drives is failing, and I'd like to
>> remove/replace it. The drive has been causing some issues (such as
>> becoming non-responsive and hanging the system with timeouts), so I'd
>> like to offline it, and then run in degraded mode until I can grab a
>> new drive (tomorrow). However, when I disconnected the drive (pulled
>> the plug, not using a zpool offline command), the following occurred:
>>
>> NAME STATE READ WRITE CKSUM
>> storage FAULTED 0 0 1
>> raidz1 DEGRADED 0 0 0
>> ad14 ONLINE 0 0 0
>> ad6 UNAVAIL 0 0 0
>> ad12 ONLINE 0 0 0
>> ad4 ONLINE 0 0 0
>>
>> Note: That's my recreation of the output... not the actual text.
>>
>> At this point, I was unable to to do anything with the pool... and all
>> data was inaccessible. Fortunately, the after sitting pulled for a
>> bit, I tried putting the failing drive back into the array, and it
>> booted properly. Of course, I still want to replace it, but this is
>> what happens when I try to take it offline:
>>
>> monolith# zpool status storage
>> pool: storage
>> state: ONLINE
>> scrub: none requested
>> config:
>>
>> NAME STATE READ WRITE CKSUM
>> storage ONLINE 0 0 0
>> raidz1 ONLINE 0 0 0
>> ad14 ONLINE 0 0 0
>> ad6 ONLINE 0 0 0
>> ad12 ONLINE 0 0 0
>> ad4 ONLINE 0 0 0
>>
>> errors: No known data errors
>> monolith# zpool offline storage ad6
>> cannot offline ad6: no valid replicas
>
> Could you send the output of:
>
> # apply "zdb -l /dev/%1" ad{4,6,12,14}
>
Sure thing.
-------------- next part --------------
--------------------------------------------
LABEL 0
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=6504613440151659848
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 1
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=6504613440151659848
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 2
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=6504613440151659848
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 3
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=6504613440151659848
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 0
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=12485416462231715770
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 1
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=12485416462231715770
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 2
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=12485416462231715770
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 3
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=12485416462231715770
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 0
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=6817203917234297503
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 1
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=6817203917234297503
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 2
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=6817203917234297503
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 3
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=6817203917234297503
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 0
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=4699988331335899116
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 1
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=4699988331335899116
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 2
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=4699988331335899116
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
--------------------------------------------
LABEL 3
--------------------------------------------
version=13
name='storage'
state=0
txg=3688995
pool_guid=5578023476090267517
hostid=2292452270
hostname='monolith'
top_guid=16261364677366705650
guid=4699988331335899116
vdev_tree
type='raidz'
id=0
guid=16261364677366705650
nparity=1
metaslab_array=14
metaslab_shift=32
ashift=9
asize=6001188143104
is_log=0
children[0]
type='disk'
id=0
guid=4699988331335899116
path='/dev/ad14'
whole_disk=0
DTL=337
children[1]
type='disk'
id=1
guid=12485416462231715770
path='/dev/ad6'
whole_disk=0
DTL=336
children[2]
type='disk'
id=2
guid=6817203917234297503
path='/dev/ad12'
whole_disk=0
DTL=335
children[3]
type='disk'
id=3
guid=6504613440151659848
path='/dev/ad4'
whole_disk=0
DTL=334
More information about the freebsd-fs
mailing list