Vinum subdisk crash

[Gareth Bailey]

Hello everyone,

Just to add, I had to type out the above message manually since I can't get 
access to anything with the crashed subdisk on /usr.
With regard to Greg's requests for information when reporting vinum problems 
as stated on vinumvm.org <http://vinumvm.org>'s website, I can provide the 
following info:

What problems are you having?
My usr.p0.s0 subdisk reports a 'crashed' status

Which version of FreeBSD are you running?
4.10 Stable

Have you made any changes to the system sources, including Vinum?
No

Supply the output of the *vinum list* command. If you can't start Vinum, 
supply the on-disk configuration, as described below. If you can't start 
Vinum,
then (and only then) send a copy of the configuration file.
I can't get anything off the system, and its too long to type out! (I have 
the same layout as in the Van Valzah article.)

Supply an *extract* of the Vinum history file. Unless you have explicitly 
renamed it, it will be */var/log/vinum_history*. This file can get very big; 
please limit it to the time around when you have the problems. Each line 
contains a timestamp at the beginning, so you will have no difficulty in 
establishing which data is of relevance.
I will summarise the tail of vinum_history (doesn't seem to provide any 
interesting info):
30 Jun 2005 ***vinum started***
30 Jun 2005 list
30 Jun 2005 ***vinum started***
30 Jun 2005 dumpconfig

Supply an *extract* of the file */var/log/messages*. Restrict the extract to 
the same time frame as the history file. Again, each line contains a 
timestamp at the beginning, so you will have no difficulty in establishing 
which data is of relevance.
Again, I will summarise the tail contents of messages:
30 Jun server /kernel: ad0s1h: hard error reading fsbn 59814344 of 
29126985-29127080 (ad0s1 bn 59814344; cn 3723 tn 69 sn 2) trying PIO mode
30 Jun server /kernel: ad0s1h: hard error reading fsbn 59814344 of 
29126985-29127080 (ad0s1 bn 59814344; cn 3723 tn 69 sn 2) status=59 error=40
30 Jun server /kernel: vinum: usr.p0.s0 is crashed by force
30 Jun server /kernel: vinum: usr.p0 is faulty
30 Jun server /kernel: vinum: usr is down
30 Jun server /kernel: fatal:usr.p0.s0 read error, block 29126985 for 49152 
bytes
30 Jun server /kernel: usr.p0.s0: user buffer block 28102720 for 49152 bytes

If you have a crash, please supply a backtrace from the dump analysis as 
discussed below under Kernel Panics. Please don't delete the crash dump; it 
may be needed for further analysis.
I'm not sure if a kernel panic occurred?

I hope this information helps, and that someone can give me some advice! 

Cheers,
Gareth

On 6/30/05, Gareth Bailey <gjbailey at gmail.com> wrote:
> 
> Hello,
> 
> It appears that one of the vinum subdisks on my server has crashed. On 
> rebooting I get the following message:
> 
> 
> <-- start message -->
> 
> Warning: defective objects
> 
> V usr State:down Plexes:2 Size:37GB
> P usr.p0 C State:faulty Subdisks:1 Size:37GB
> P usr.p1 C State:faulty Subdisks:1 Size:37GB
> S usr.p0.s0 State:crashed P0:0 B Size:37GB
> S usr.p1.s0 State:stale P0:0 B Size:37GB
> 
> [some fsck messages]
> Can't open /dev/vinum/usr: Input/output error
> [some more fsck messages]
> THE FOLLOWING FILE SYSTEM HAD AN UNEXPEXTED INCONSISTENCY:
> /dev/vinum/usr (/usr)
> Automatic file system checj failed . . . help!
> Enter full pathname of shell....
> 
> <-- end message -->
> 
> I have a straight forward configuration based on the "Bootstrapping Vinum: 
> A Foundation for Reliable Servers" article by Robert Van Valzah.
> What could have caused this? The disks are pretty new. Please advise on 
> the quickest route to getting our server back online.
> 
> Much appreciated,
> 
> Gareth Bailey