geli problems after installkernel & installworld

Sun Jan 16 14:56:07 UTC 2011

On Sun, 16 Jan 2011 14:24:58 +0100
Fabian Keil <freebsd-listen at fabiankeil.de> wrote:

> "Christopher J. Ruwe" <cjr at cruwe.de> wrote:
> 
> > On Sat, 15 Jan 2011 22:30:56 +0100
> > Pawel Jakub Dawidek <pjd at FreeBSD.org> wrote:
> > 
> > > On Thu, Jan 13, 2011 at 10:00:19PM +0100, Christopher J. Ruwe
> > > wrote:
> > > > I use a mostly geli encrypted hd on my Thinkpad R500,
> > > > with /compat, /usr, /tmp and /var all on the encrypted geli
> > > > provider.
> > > > 
> > > > After an upgrade of kernel and world (STABLE), I experience a
> > > > weird issue: While booting, I am asked for the geli passphrase
> > > > as usual. Completing password authentication for geli returns a
> > > > success message,
> > > > 
> > > > cryptosoft0: <software crypto> on motherboard
> > > > GEOM_ELI: Device ada0p3.eli created.
> > > > GEOM_ELI: Encryption: AES-CBC 256
> > > > GEOM_ELI:     Crypto: software
> > > > 
> > > > however, the zpool on geli is unavailable.
> > > > 
> > > > Logging in a root, I can attach the geli provider manually as
> > > > geli itself should do from /etc/rc.conf. After a successful zfs
> > > > mount -a, I can resume as usual after manually starting
> > > > the /usr/local/rc.d services. 
> > > > 
> > > > Neither have I noticed a change in the device names nor any
> > > > unusual messages from dmesg. Currently, I am doing a new
> > > > compile run on world and kernel to attempt anew tomorrow.
> > > > 
> > > > Am I missing something?
> > > 
> > > Can you show the output of 'geli list' from a running system?
> > > 
> > 
> > Sure I can ... I'll additionally  comment the output with what I do
> > to.
> > 
> > First I boot and my /usr/local/rc.d/ - schripts do not start.
> > Likewise does zsh.
> > 
> > From doing geli list, I get (on stdout)
> > 
> > Geom name: ada0p3.eli
> > State: ACTIVE
> > EncryptionAlgorithm: AES-CBC
> > KeyLength: 256
> > Crypto: software
> > UsedKey: 0
> > Flags: SINGLE-KEY, NATIVE-BYTE-ORDER, BOOT, RW-DETACH
> > Providers:
> > 1. Name: ada0p3.eli
> >    Mediasize: 249656594432 (233G)
> >    Sectorsize: 4096
> >    Mode: r0w0e0
> > Consumers:
> > 1. Name: ada0p3
> >    Mediasize: 249656596992 (233G)
> >    Sectorsize: 512
> >    Mode: r1w1e1
> > 
> > Doing a zpool status -v gives on stdout
> > 
> >  pool: ntank
> >  state: UNAVAIL
> > status: One or more devices could not be opened.  There are
> > insufficient replicas for the pool to continue functioning.
> > action: Attach the missing device and online it using 'zpool
> > online'. see: http://www.sun.com/msg/ZFS-8000-3C
> >  scrub: none requested
> > config:
> > 
> >         NAME          STATE     READ WRITE CKSUM
> >         ntank         UNAVAIL      0     0     0  insufficient
> > replicas ada0p3.eli  UNAVAIL      0     0     0  cannot open
> > 
> >   pool: rpool
> >  state: ONLINE
> > status: The pool is formatted using an older on-disk format.  The
> > pool can still be used, but some features are unavailable.
> > action: Upgrade the pool using 'zpool upgrade'.  Once this is done,
> > the pool will no longer be accessible on older software versions.
> >  scrub: none requested
> > config:
> > 
> >         NAME                                          STATE     READ
> >         WRITE CKSUM rpool
> >         ONLINE       0     0     0
> >         gptid/3ab00705-d22f-11df-8e1b-002713b40a7b  ONLINE       0
> >         0     0
> > 
> > errors: No known data errors
> > 
> > and on stderr ( I noticed the output on stderr as I ran the
> > command, so I just typed that)
> > 
> > GEOM_ELI[1]: Device ada0p3.eli is still open, so it cannot be
> > definitely removed.
> > GEOM_ELI[1]: Detached ada0p3.eli on last close.
> > 
> > When doing a geli attach -k /pathtomykey/key /dev/ada0p3 directly
> > followed by a zfs mount -a, I have my filesystems where I am used to
> > finding them. I run my /usr/local/rc.ds from there and am functional
> > again.
> > 
> > Then (I post this anwe, I will point out why later on), I get for
> > geli list
> > 
> > Geom name: ada0p3.eli
> > State: ACTIVE
> > EncryptionAlgorithm: AES-CBC
> > KeyLength: 256
> > Crypto: software
> > UsedKey: 0
> > Flags: SINGLE-KEY, NATIVE-BYTE-ORDER, BOOT
> > Providers:
> > 1. Name: ada0p3.eli
> >    Mediasize: 249656594432 (233G)
> >    Sectorsize: 4096
> >    Mode: r1w1e1
> > Consumers:
> > 1. Name: ada0p3
> >    Mediasize: 249656596992 (233G)
> >    Sectorsize: 512
> >    Mode: r1w1e1
> > 
> > I never noticed that before, but, as I did not know which geli
> > output you were asking for (the one not working or the one
> > working), I diffed the two files and noticed, that directly  after
> > booting, the RW-DETACH flag is set. I do not know what that means
> > nor do I know whether that matters, I find that curious, though.
> 
> I'm no sure if it's the cause of your problem,
> but it certainly does matter:
> http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/117158
> 
> Fabian

Thank you. What you mentioned apparently was part of my problem. After
setting geli_autodetach="no", at least the geli part of my problem goes
away. This is most interesting as I did not change anything on my
geli-setup, so I believe behaviour might have changed somewhere, either
in rc.defaults or somewhere else.

Another interesting point, although the zpools sit right where and how
they belong, the fs on the pool sitting on the geli still need manually
mounting, which requires manually zfs mount -a and then manually
the /usr/local/rc.d/ . I can live with that for some time, though,
until I have found a solution for that. I could easily (and just as
dirtily) solve that with some rc.local line, but that probably won't
help me with my /usr/local/rc.d stuff.

Again, thank you for your hint, cheers,
-- 
Christopher J. Ruwe
TZ GMT + 1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 834 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20110116/89283758/signature.pgp