zpool scrub on pool from geli devices offlines the pool?

[Nikolay Denev]

On Oct 4, 2012, at 11:24 PM, Fabian Keil <freebsd-listen at fabiankeil.de> wrote:

> Nikolay Denev <ndenev at gmail.com> wrote:
> 
>> I have a zfs pool from 24 disks encrypted with geli.
>> 
>> I just did a zpool scrub tank, and that probably reopened all of the devices,
>> but this caused geli "detach on last close" to kick in 
>> which resulted in offline pool from UNAVAILABLE devices. 
> 
> This is a known issue:
> http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/117158
> 
> The fact that the system didn't panic seems like an improvement,
> although this might be the result of the different pool layout.
> 
>>  pool: tank
>> state: UNAVAIL
>> status: One or more devices are faulted in response to IO failures.
>> action: Make sure the affected devices are connected, then run 'zpool clear'.
>>   see: http://illumos.org/msg/ZFS-8000-HC
>>  scan: scrub in progress since Thu Oct  4 21:19:15 2012
>>        1 scanned out of 8.29T at 1/s, (scan is slow, no estimated time)
>>        0 repaired, 0.00% done
>> config:
>> 
>> 	NAME                      STATE     READ WRITE CKSUM
>> 	tank                      UNAVAIL      0     0     0
> [...]
>> 
>> errors: 1 data errors, use '-v' for a list
>> 
>> Dmesg shows :
>> 
>> GEOM_ELI: Detached mfid1.eli on last close.
>> …
>> GEOM_ELI: Detached mfid24.eli on last close.
>> 
>> I then did /etc/rc.d/geli restart and zpool clear tank, and it is back online,
>> but shows permanent metadata errors…
> 
> I'd expect the "permanent" metadata errors to be gone after
> the scrubbing is completed.
> 
>> Any ideas why this happned from a simple zpool scrub, and how it can be prevented?
>> Just disable "detach on last close" for the geli devices?
> 
> At least that was Pawel's recommendation in 2007:
> http://lists.freebsd.org/pipermail/freebsd-current/2007-October/078107.html
> 
> Fabian

Thanks for the information, I have missed that.

And yep, the pool reports as ONLINE without errors after the reboot.
I'll add geli_autodetach="NO" to rc.conf.

Regards,
Nikolay