Problem with zpool remove of log device

[Łukasz Wąsikowski]

W dniu 2017-06-15 o 13:54, Stephen McKay pisze:

> On Friday, 9th June 2017, lukasz at wasikowski.net wrote:
> 
>> W dniu 2017-06-09 o 05:18, Stephen McKay pisze:
>>
>>>     while :
>>>     do
>>>         date > foo
>>>         fsync foo
>>>     done
>>>
>>> With this running, my system does 600 writes per second to the log
>>> according to zpool iostat.  That drops to zero once I kill the script.
>>
>> Zero, so no writes to log are performed during execution of this script.
> 
> OK.  I believe this means your log is in a "pending removal" state and
> has not been finally removed because ZFS thinks there's still data
> stored on it.  I'm happy for true ZFS experts to confirm or deny this
> theory.  Anybody?
> 
>> I applied this patch to 11.1-PRERELASE, nothing changed. Still zpool
>> remove exits with errcode 0, but log device is still attached to pool.
> 
> Thanks for trying this out, but I managed to leave out essential
> information.
> 
> In my rush to do things before going away (see above), I didn't read my
> notes on this event.  The patch has the safety feature of requiring the
> log to be offline.  This means you will have to break the mirror (by
> detaching one disk from it), then offline the remaining disk, and finally,
> trigger the hack by attempting to remove the remaining disk.
> 
> When I was stuck in this situation, I had already reduced my log to
> a single disk before discovering the accounting error problem, so I
> don't know if you can activate the patch code without first breaking
> the mirror.  I don't think you can offline a mirror.  I've not tried.
> 
> I've now had time to review my notes (sorry I didn't do that first up).
> My pool had a data mirror-0 (gpt/data0 and gpt/data1) and a log mirror-1
> (gpt/log0 and gpt/log1).  The sequence I did, minus most of the failed
> attempts at random stuff, status checks, and so forth, was:
> 
> # zpool remove pool mirror-1   #Did nothing but should have worked.
> # zpool detach pool gpt/log1   #Broke the log mirror.
> # zpool remove pool gpt/log0   #Did nothing but should have worked.
> # zpool offline pool gpt/log0  #Just fiddling to change the state.
> # zpool remove pool gpt/log0   #Still nothing.
> ... Discovered plausible hack. Built and booted hacked kernel. ...
> # zpool remove pool gpt/log0   #Glorious success!
> 
> So, my log was already down to one offline disk before I got hacky.
> That's why I forgot to mention it.
> 
> You could break your mirror or you could modify the hack to remove the
> VDEV_STATE_OFFLINE check.  If you have already saved all your important
> data then either would be fine as an experiment.

My tests was done after zpool remove pool mirror-1, zpool detach pool
gpt/log1. I don't remember if I offlined gpt/log0. It's possible that I
haven't done it. Bulit and booted hacked kernel, zpool remove pool
gpt/log0 didn't work.

Unfortunately, lease of this box ended 14.06, I zeroed drives on 13.06 :(

Again, thank you for your help!

-- 
best regards,
Lukasz Wasikowski