zpool scrub on pool from geli devices offlines the pool?

[Fabian Keil]

Nikolay Denev <ndenev at gmail.com> wrote:

> I have a zfs pool from 24 disks encrypted with geli.
> 
> I just did a zpool scrub tank, and that probably reopened all of the devices,
> but this caused geli "detach on last close" to kick in 
> which resulted in offline pool from UNAVAILABLE devices. 

This is a known issue:
http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/117158

The fact that the system didn't panic seems like an improvement,
although this might be the result of the different pool layout.

>   pool: tank
>  state: UNAVAIL
> status: One or more devices are faulted in response to IO failures.
> action: Make sure the affected devices are connected, then run 'zpool clear'.
>    see: http://illumos.org/msg/ZFS-8000-HC
>   scan: scrub in progress since Thu Oct  4 21:19:15 2012
>         1 scanned out of 8.29T at 1/s, (scan is slow, no estimated time)
>         0 repaired, 0.00% done
> config:
> 
> 	NAME                      STATE     READ WRITE CKSUM
> 	tank                      UNAVAIL      0     0     0
[...]
> 
> errors: 1 data errors, use '-v' for a list
> 
> Dmesg shows :
> 
> GEOM_ELI: Detached mfid1.eli on last close.
> …
> GEOM_ELI: Detached mfid24.eli on last close.
> 
> I then did /etc/rc.d/geli restart and zpool clear tank, and it is back online,
> but shows permanent metadata errors…

I'd expect the "permanent" metadata errors to be gone after
the scrubbing is completed.

> Any ideas why this happned from a simple zpool scrub, and how it can be prevented?
> Just disable "detach on last close" for the geli devices?

At least that was Pawel's recommendation in 2007:
http://lists.freebsd.org/pipermail/freebsd-current/2007-October/078107.html

Fabian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20121004/8edf8692/signature.pgp