GELI + Zpool Scrub Results in GELI Device Destruction (and Later a Corrupt Pool)

[Fabian Keil]

"Michael B. Eichorn" <ike at michaeleichorn.com> wrote:

> On Mon, 2016-04-25 at 10:11 +0200, Fabian Keil wrote:
> > "Michael B. Eichorn" <ike at michaeleichorn.com> wrote:
> >   
> > > 
> > > I just ran into something rather unexpected. I have a pool
> > > consisting
> > > of a mirrored pair of geli encrypted partitions on WD Red 3TB
> > > disks.
> > > 
> > > The machine is running 10.3-RELEASE, the root zpool was setup with
> > > GELI
> > > encryption from the installer, the pool that is acting up was setup
> > > per
> > > the handbook.  
> > [...]  
> > > 
> > > I had just noticed that I had failed to enable the zpool scrub
> > > periodic
> > > on this machine. So I began to run zpool scrub by hand. It
> > > succeeded
> > > for the root pool which is also geli encrypted, but when I ran it
> > > against my primary data pool I encountered:
> > > 
> > > Apr 24 23:18:23 terra kernel: GEOM_ELI: Device ada3p1.eli
> > > destroyed.
> > > Apr 24 23:18:23 terra kernel: GEOM_ELI: Detached ada3p1.eli on last
> > > close.
> > > Apr 24 23:18:23 terra kernel: GEOM_ELI: Device ada2p1.eli
> > > destroyed.
> > > Apr 24 23:18:23 terra kernel: GEOM_ELI: Detached ada2p1.eli on last
> > > close.  
> > Did you attach the devices using geli's -d (auto-detach) flag?
> 
> I am using whatever the default setup comes out of the rc.d scripts.
> My rc.conf was:
> 
> geli_devices="ada2p1 ada3p1"
> geli_default_flags="-k /root/encryption.key"
> zfs_enable="YES"
> 
> I will try adding geli_autodetach="NO" and scubbing in about 9 hours.

On FreeBSD the default (set in /etc/defaults/rc.conf) is YES.

> > If yes, this is a known issue:
> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=117158
> >   
> 
> Reading that bug in detail it appears to be *specifically* for the
> kernel panic and that zfs closing and reopening providers is expected
> behavior, and that if geli has autodetach configured that it would
> detach.
> 
> It stikes me that even though this is expected behavior it should not
> be. Is there a way we could prevent the detach when zfs does closes and
> reopens providers? I cannnot think of a case where the desired behavior
> is for the pool to detach when zfs wants to reopen it.

I suppose geli could delay the detachment a bit to give the consumer
a chance to reopen it.

Fabian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20160425/3821a25e/attachment.sig>