Vinum: after kernel panic, incorrect avail shown on drives (long)
Tillman
tillman at seekingfire.com
Tue May 6 10:41:44 PDT 2003
(Warning: long and verbose message. I'm following the directions on
http://www.vinumvm.org/, so there's lots of details here).
Quick problem summary
=====================
`vinum ld` reports 5120M of available free space on a drive that is
actually full.
This occurred after I made two sets of mistakes while trying to expand a
volume (something I still find complex, apparantly). The details of the
mistakes are given as paragraph above the 'vinum history' below.
Vinum appears to be working correctly, and all filesystems pass a fsck
check (required, as the computer went down dirty). I'm worried about the
inconsistency, though, as it doesn't bode well for the future.
Version info
============
I'm running FreeBSD 4.8-STABLE (Tue Apr 29 10:26:03 CST 2003), with no
changes to kernel or vinum sources.
Confirming that the drive really is full
========================================
I've confirmed that the drive is, indeed, full by using `vinum
printconfig | grep yin` ("yin" is one of the two drives in the mirror, no
points for guessing the other drives name) to get teh names of the
subdisks that are on one of those identical drives and then running:
vinum ls mp3.p0.s0 iso.p0.s0 tillman.p0.s0 pub.p0.s0 syslog.p0.s0 \
www.p0.s0 ad.p0.s0 backups.p0.s0 usr.obj.p0.s0 iso.p0.s0 \
www.rw.p0.s0 named.p0.s0 usr.local.p0.s0 | \
awk -v ORS=" + " '{print $9}END{print "0\n"}' | bc
Which results in "38172". Comparing that to `vinum ld yin`:
D yin State: up Device /dev/ad0s1e Avail: 5120/38172 MB (13%)
Thus 38172 should represent a full drive.
vinum list (warning: wide)
==========================
[root at athena root]# vinum list
4 drives:
D yin State: up Device /dev/ad0s1e Avail: 5120/38172 MB (13%)
D yang State: up Device /dev/ad2s1e Avail: 5120/38172 MB (13%)
D heaven State: up Device /dev/da3s1e Avail: 20172/44876 MB (44%)
D earth State: up Device /dev/da4s1e Avail: 20172/44876 MB (44%)
17 volumes:
V mp3 State: up Plexes: 2 Size: 5120 MB
V iso State: up Plexes: 2 Size: 5596 MB
V tillman State: up Plexes: 2 Size: 5120 MB
V pub State: up Plexes: 2 Size: 4096 MB
V syslog State: up Plexes: 2 Size: 5120 MB
V www State: up Plexes: 2 Size: 1024 MB
V ad State: up Plexes: 2 Size: 9216 MB
V backups State: up Plexes: 2 Size: 5120 MB
V usr.obj State: up Plexes: 2 Size: 1024 MB
V tillman.backup1 State: up Plexes: 1 Size: 5120 MB
V www.rw State: up Plexes: 2 Size: 500 MB
V named State: up Plexes: 2 Size: 10 MB
V usr.local State: up Plexes: 2 Size: 1346 MB
V homes State: up Plexes: 2 Size: 10 GB
V ports State: up Plexes: 2 Size: 4096 MB
V tillman.backup2 State: up Plexes: 1 Size: 5120 MB
V cache State: up Plexes: 2 Size: 128 MB
34 subdisks:
S mp3.p0.s0 State: up PO: 0 B Size: 5120 MB
S mp3.p1.s0 State: up PO: 0 B Size: 5120 MB
S iso.p0.s0 State: up PO: 0 B Size: 4096 MB
S iso.p1.s0 State: up PO: 0 B Size: 4096 MB
S tillman.p0.s0 State: up PO: 0 B Size: 5120 MB
S tillman.p1.s0 State: up PO: 0 B Size: 5120 MB
S pub.p0.s0 State: up PO: 0 B Size: 4096 MB
S pub.p1.s0 State: up PO: 0 B Size: 4096 MB
S syslog.p0.s0 State: up PO: 0 B Size: 5120 MB
S syslog.p1.s0 State: up PO: 0 B Size: 5120 MB
S www.p0.s0 State: up PO: 0 B Size: 1024 MB
S www.p1.s0 State: up PO: 0 B Size: 1024 MB
S ad.p0.s0 State: up PO: 0 B Size: 4096 MB
S ad.p1.s0 State: up PO: 0 B Size: 4096 MB
S backups.p0.s0 State: up PO: 0 B Size: 5120 MB
S backups.p1.s0 State: up PO: 0 B Size: 5120 MB
S usr.obj.p0.s0 State: up PO: 0 B Size: 1024 MB
S usr.obj.p1.s0 State: up PO: 0 B Size: 1024 MB
S iso.p0.s1 State: up PO: 4096 MB Size: 1500 MB
S iso.p1.s1 State: up PO: 4096 MB Size: 1500 MB
S www.rw.p0.s0 State: up PO: 0 B Size: 500 MB
S www.rw.p1.s0 State: up PO: 0 B Size: 500 MB
S named.p0.s0 State: up PO: 0 B Size: 10 MB
S named.p1.s0 State: up PO: 0 B Size: 10 MB
S usr.local.p0.s0 State: up PO: 0 B Size: 1346 MB
S usr.local.p1.s0 State: up PO: 0 B Size: 1346 MB
S homes.p0.s0 State: up PO: 0 B Size: 10 GB
S homes.p1.s0 State: up PO: 0 B Size: 10 GB
S ports.p0.s0 State: up PO: 0 B Size: 4096 MB
S ports.p1.s0 State: up PO: 0 B Size: 4096 MB
S tillman.backup1.p0.s0 State: up PO: 0 B Size: 5120 MB
S tillman.backup2.p0.s0 State: up PO: 0 B Size: 5120 MB
S cache.p0.s0 State: up PO: 0 B Size: 128 MB
S cache.p1.s0 State: up PO: 0 B Size: 128 MB
S ad.p0.s1 State: up PO: 4096 MB Size: 5120 MB
S ad.p1.s1 State: up PO: 4096 MB Size: 5120 MB
vinum history
=============
Note that vinum10.conf was first name of the config file I was intended
to use with 'vinum create -f'. My first mistake (besides having having a
completely wrong approach to growing volumes) was that I forgot to
change the volume name (and which resulted in my first ever kernel
panic). Next, I had edited it show the correct volume name (and
neglected to figure out that I still didn't want to create a new volume,
just make new subdisks and attach them to the existing volume). Lastly,
vinum10.conf is the working config file where I had things correct.
Third try's the charm, I guess :-)
5 May 2003 18:45:01.977785 *** vinum started ***
5 May 2003 18:45:06.788381 create -f /etc/vinum10.conf
volume cache setupstate
plex org concat
sd length 5120m drive heaven
plex org concat
sd length 5120m drive earth
5 May 2003 18:45:06.790121 *** Created devices ***
5 May 2003 18:45:36.382621 help
5 May 2003 18:48:21.229082 *** vinum started ***
5 May 2003 18:48:21.864657 list
5 May 2003 18:48:32.012001 rm -rf cache.p2
5 May 2003 18:48:33.705788 rm -rf cache.p3
5 May 2003 18:48:34.884671 list
5 May 2003 18:48:40.700523 exit
5 May 2003 18:48:46.619180 create -f /etc/vinum10.conf
volume ad setupstate
plex org concat
sd length 5120m drive heaven
plex org concat
sd length 5120m drive earth
5 May 2003 18:48:46.620862 *** Created devices ***
5 May 2003 18:49:46.928880 rm -rf ad.p2.s0
5 May 2003 13:04:04.116544 *** vinum started ***
5 May 2003 13:04:04.163722 start
5 May 2003 13:04:08.148507 *** Created devices ***
5 May 2003 19:14:10.810116 *** vinum started ***
5 May 2003 19:14:11.546075 list
5 May 2003 19:14:22.073064 rm -rf ad.p2
5 May 2003 19:14:25.738567 rm -rf ad.p3
5 May 2003 19:14:26.553543 list
5 May 2003 19:14:33.809775 lv
5 May 2003 19:14:39.222651 lp
5 May 2003 19:14:52.711902 quit
5 May 2003 19:17:24.060960 *** vinum started ***
5 May 2003 19:17:25.979662 lp
5 May 2003 19:17:26.560207 ls
5 May 2003 19:17:37.935742 quit
5 May 2003 19:19:15.977739 *** vinum started ***
5 May 2003 19:19:15.978251 ls
5 May 2003 19:19:21.942660 *** vinum started ***
5 May 2003 19:19:48.708374 list
5 May 2003 19:19:57.650730 create -f /etc/vinum10.conf
subdisk name ad.p0.s1 length 5120m plex ad.p0 drive heaven
subdisk name ad.p1.s1 length 5120m plex ad.p1 drive earth
5 May 2003 19:19:57.652350 *** Created devices ***
5 May 2003 19:20:14.993509 lv
5 May 2003 19:20:26.257590 quit
5 May 2003 19:24:11.059268 *** vinum started ***
The matching time in /var/log/messages appears to be:
May 5 18:48:46 athena /kernel: vinum: ad.p2 is faulty
May 5 18:48:46 athena /kernel: vinum: ad.p3 is faulty
May 5 18:49:47 athena /kernel: vinum: removing ad.p2.s0
May 5 19:09:49 athena /kernel: Copyright (c) 1992-2003 The FreeBSD Project.
Looks like the kernel panic was right around 18:49:48.
TIA for any help,
- Tillman
--
"The gods too, are fond of a joke."
- Aristotle
More information about the freebsd-questions
mailing list