zfs fails to mount correctly during 8.2 -> 9.0 update

[krad]

On 2 March 2012 14:44, Volodymyr Kostyrko <c.kworr at gmail.com> wrote:

> Hi all.
>
> I'm writing it just in case someone else will face this problem.
>
> Some days ago I was moving my last server from 8.2 to 9.0. At first I had
> installed kernel with:
>
>  make KODIR=/boot/test installkernel
>
> and rebooted. At loader prompt I selected this kernel with:
>
>  unload
>  boot-conf /boot/test
>
> After that 9.0 kernel was loaded with 8.2 userland. I know this is not the
> right-and-only-one way of updating FreeBSD but it works for me through
> releases. But this time something strange happened. During system mount
> kernel spits out something like:
>
>  Solaris: WARNING: metaslab_free_dva(): bad DVA 0:52834975928475
>
> It was something like one page of this lines.
>
> The pool seems to be mounted readonly and refused to remount writable. It
> return zero empty space and status stated something about stopped scrub.
> Second pull was writable and working. However I haven't updated the
> userland so I just rebooted back to 8.2 kernel.
>
> Machine comes back online at full health. Everything stated both pools are
> healthy and working. Scrub doesn't yield any results - not a single error.
>
> I wiped /usr/obj and rebuilt everything from a scratch. After planting a
> test kernel and booting from it everything worked fine so I continued with
> upgrade. After upgrade was committed and pools were updated to ZFSv28 I
> repeated scrub on both. There was no single error.
>
> --
> Sphinx of black quartz judge my vow.
> ______________________________**_________________
> freebsd-fs at freebsd.org mailing list
> http://lists.freebsd.org/**mailman/listinfo/freebsd-fs<http://lists.freebsd.org/mailman/listinfo/freebsd-fs>
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@**freebsd.org<freebsd-fs-unsubscribe at freebsd.org>
> "
>


I have generally found this stepped method of booting into the new kernel
with the old userland as a bad thing with zfs. Especially if there is a zfs
version bump in the new build. This is due the the user land zfs tools
being out of sync with the kernel. What I have found works much better is
to clone your zfs root file system, mount it somewhere, set DESTDIR to
there and install world and kernel. Then tweak the loader.conf on the
relevent fs and the pools bootfs property accordingly.  Unmount the new
root fs and set the mountpoint the legacy, then reboot. If you get problems
with the new build just boot in with a live ios and reset the bootfs
property of the pool to the old value. I wish the last step could be done
from the loader prompt but as far as i can see you cant yet 8(