volumes crash on reboot [was Re: clearing Vinum configurations]

[aarong]

After some extensive testing, I've come to the conclusion that Vinum 
refuses to create any type of volume on the drive I'm having trouble 
with. I'll create any number of volumes and/or additional plexes on the 
second drive in the system successfully, only to have all the volumes 
or plexes that reside on the second drive crash on reboot. Vinum's logs 
in /var/log/vinum_history aren't very helpful nor are any of the other 
debugging techniques listed in the vinum debug howto.

What I have not been able to find is a way to understand why Vinum 
declares a volume crashed or a plex faulty. I've initialized and 
reinitialized the second drive until I was blue in the face; creating 
filesystems and fsck'ing them; copying large amounts of to arbitrarily 
created slices all without incident. Only Vinum has a problem with the 
second drive, and only upon a reboot of the machine. I've had no issue 
adding plexes to existing volumes and mirroring them.

/var/log/vinum_history is filled with "vinum started", "list", 
"dumpconfig", "start usr.p1", and "quit" messages - nothing descriptive 
of interest. The same goes for "vinum list" output, as well as dd'ing 
the sixth sector on both drives. This is a remote box without 
-DVINUMDEBUG built and such a procedure is far too prohibitive; it's 
already taken a week to find a competent tech at the datacenter to work 
with Vinum and another week to teach him how to setup a bootable Vinum 
volume. Needless to say we're far behind schedule. As much as I'd like 
to mirror things and leave it at that, it worries me greatly that 
something could be physically wrong with the second hard drive and this 
software RAID 1 setup is only giving my client a false sense of 
security. I'll need some sort of concrete evidence to ask for a new 
hard drive to be installed.

The first drive, known to Vinum as "alpha", is ad0 and has four perfect 
volumes which the system runs off of. The second drive, known to Vinum 
as "beta", is the slave on the secondary channel and hence is device 
ad3. I'm almost sure this has no relevance but as you can tell I'm 
completely lost. Both are identical in model, revision, and size.

Regards,
-aarong