[SOLVED] installkernel succeeds, but old kernel boots

[Paco Hope]

In case anyone is curious, here's the answer.  The information needed to
solve this wasn't in my email.  Look in the archives for the original
problem description.

The short and amusing answer: it was booting off a stale vinum drive, and
then running off the current drive once vinum got loaded.

Here's the long answer.

I'm one of the unlucky ones who was using vinum in 5.1 and 5.2 only to
discover that it broke in 5.3.  I am booting off a vinum root, too.  I
have the a partition adjusted on both root drives so that it appears to be
a normal root partition, but then the d partition is the real vinum
partition.

Well, here's what was going on.  At boot time I would see:

kernel: vinum: incompatible sector sizes.  mroot.p1 has 0, mroot has 512.  Ignored.

My root vinum volume ("mroot") would come up OK, but one of its plexes
(mroot.p1) would be down because its only subdisk was missing.  For some
reason, it would never see the config in /dev/ad1s1, it would never find
the drive on /dev/ad1s1d and thus never start up the mroot.p1 plex.

If I interactively ran vinum and typed "read /dev/ad1s1" it would find all
the drives that are on ad1s1.  I could individually start subdisks, which
would refresh and life would be happy for a while.  I was seeing some
kernel panics, though, so I eventually quit manually starting up these
drives.  If I didn't actively mirror, I didn't have panics.

Well, my 'make installkernel' was installing onto the live, active mirror
of my root filesystem.  For some reason, though, the boot selector on the
console was trying (and succeeding) booting the second disk.  Since it
wasn't up-to-date, it still had a legit filesystem on it, but not one that
had the right kernel.  My 5.3 kernel would boot, load the vinum module,
switch the root partition to the vinum root, and then run.  I was
dumbfounded at how the file I saw as /boot/kernel/kernel was not the
kernel I booted.

So I ran vinum and manually started up the subdisk that corresponded to my
root partition.  It synchronized.  Then I rebooted.  Two things are true
now that make me happy:

1. I'm booting my 5.4-STABLE kernel.
2. vinum seems to be working.  No panics yet and I no longer have this
problem where the ad1s1 vinum partitions are lost at boot time.  If I get
time, I will probably switch to gmirror for mirroring these partitions,
but I feel less pressure to do so, now that this seems to be working
again.

I hope this explanation helps someone.


Paco