vinum raid5 subdisks keep changing length?

Tony Frank tfrank at optushome.com.au
Mon Feb 16 03:04:48 PST 2004


Hi all,

Running vinum on RELENG_4.

No changes to source, vinum loaded as a KLD as per instructions in handbook &
man pages.

As per the subject I am having what appears to be strange behaviour with vinum.

After a number of teething problems I eventually setup a vinum root for
my system which covers / swap /var and /usr.
This works just fine, though I am yet to experiment with failure scenarios.

However I also configured a raid5 volume and may be having issues with it.
Currently just a test system and no obvious data loss but I have some
concerns before I put it into production.

Specifically I originally configured five subdisks (on different drives)
with identical lengths and then combined them into a raid5 plex.
This worked ok and I successfully initialised the plex and used newfs on it.
Successfully able to read&write to the volume (1G/14G used at the moment)

The original config looked as so:

volume data
plex name data.p0 org raid5 984s vol data 
sd name data.p0.s0 drive eightgig plex data.p0 len 8802864s driveoffset 6335201s plexoffset 0s
sd name data.p0.s1 drive vinumdrive0 plex data.p0 len 8802864s driveoffset 265s plexoffset 984s
sd name data.p0.s2 drive vinumdrive1 plex data.p0 len 8802864s driveoffset 265s plexoffset 1968s
sd name data.p0.s3 drive vinumdrive2 plex data.p0 len 8802864s driveoffset 265s plexoffset 2952s
sd name data.p0.s4 drive vinumdrive3 plex data.p0 len 8802864s driveoffset 265s plexoffset 3936s

Now after each system reboot I see messages on system console from vinum.
(they are not logged in vinum_history or messages) 

(hand transcript):

vinum: updating configuration from /dev/ad0s1h 
#(repeated for all 6 drives)
vinum: removing 2560 blocks of partial stripe at the end of data.p0
vinum: data.p0 must have equal sized subdisks
vinum: data.p0 must have equal sized subdisks
vinum: data.p0 must have equal sized subdisks
vinum: data.p0 must have equal sized subdisks
Correcting length of data.p0: was 35191776, is 35197088
vinum: using volume root for root device
Mounting root from ufs:/dev/vinum/root

A "vinum printconfig" now shows difference values for the subdisks.
This value has changed after each reboot.

Current relevant parts of 'printconfig' are:

volume data
plex name data.p0 org raid5 984s vol data 
sd name data.p0.s0 drive eightgig plex data.p0 len 8799978s driveoffset 6335201s plexoffset 0s
sd name data.p0.s1 drive vinumdrive0 plex data.p0 len 8799486s driveoffset 265s plexoffset 984s
sd name data.p0.s2 drive vinumdrive1 plex data.p0 len 8799158s driveoffset 265s plexoffset 1968s
sd name data.p0.s3 drive vinumdrive2 plex data.p0 len 8798420s driveoffset 265s plexoffset 2952s
sd name data.p0.s4 drive vinumdrive3 plex data.p0 len 8797600s driveoffset 265s plexoffset 3936s

Each subdisk len is now a different size which is of some concern to me, 
especially as the specific len values change after each reboot.

Some other perhaps 'relevant' details:

uname -a output:
FreeBSD raider.home.local 4.9-STABLE FreeBSD 4.9-STABLE #1: Sat Feb  7 01:50:58 EST 2004     tony at marvin.home.local:/data/ad3/obj/usr/src/sys/RAIDER  i386

Tail of /var/log/messages output:

Feb 16 20:14:42 raider /kernel: da2: 20.000MB/s transfers (10.000MHz, offset 8, 16bit), Tagged Queueing Enabled
Feb 16 20:14:42 raider /kernel: da2: 4303MB (8813870 512 byte sectors: 255H 63S/T 548C)
Feb 16 20:14:42 raider /kernel: Mounting root from ufs:/dev/vinum/root
Feb 16 20:14:43 raider savecore: no core dump
Feb 16 20:14:37 raider ntpd[115]: ntpd 4.1.0-a Sat Feb  7 00:06:01 EST 2004 (1)
Feb 16 20:14:37 raider ntpd[115]: kernel time discipline status 2040
Feb 16 20:14:54 raider su: tony to root on /dev/ttyp0
Feb 16 20:15:03 raider su: tony to root on /dev/ttyp0
Feb 16 20:19:10 raider ntpd[115]: time set 0.007497 s

/var/log/vinum_history extract (shows init and subsequent prints)

 8 Feb 2004 13:38:58.679668 attach data.p0 data
 8 Feb 2004 13:39:01.616409 lv
 8 Feb 2004 13:39:02.891476 lv -r
 8 Feb 2004 13:39:04.099017 lv
 8 Feb 2004 13:39:09.498986 lv -r
 8 Feb 2004 13:39:17.697480 init data.p0
 8 Feb 2004 13:40:24.011586 lv -r
 8 Feb 2004 13:41:23.409672 lv -r
 8 Feb 2004 13:42:06.320951 lv -r
 8 Feb 2004 13:43:17.928172 *** vinum started ***
 8 Feb 2004 13:43:17.943327 lv -r
 8 Feb 2004 13:44:26.669794 *** vinum started ***
 8 Feb 2004 13:44:26.675611 lv -r
 8 Feb 2004 13:46:06.942916 *** vinum started ***
 8 Feb 2004 13:46:06.945251 lv -r
 8 Feb 2004 13:49:36.588518 *** vinum started ***
 8 Feb 2004 13:49:36.590516 lv -r
 8 Feb 2004 13:53:32.719039 *** vinum started ***
 # Rebooted here
 8 Feb 2004 15:26:57.032467 *** vinum started ***
 8 Feb 2004 15:26:57.053653 list
 8 Feb 2004 15:27:00.942329 *** vinum started ***
 8 Feb 2004 15:27:00.943709 printconfig
 8 Feb 2004 15:27:22.021212 *** vinum started ***
 8 Feb 2004 15:27:22.022626 printconfig after_reboot.config
 8 Feb 2004 15:29:53.377752 *** vinum started ***
 8 Feb 2004 15:29:53.379181 l
 8 Feb 2004 15:31:16.065588 *** vinum started ***
 8 Feb 2004 15:31:16.067022 ld
# Rebooted here
11 Feb 2004 23:15:51.299376 *** vinum started ***
11 Feb 2004 23:15:51.317598 printconfig latest_reboot.config
# Rebooted here
16 Feb 2004 20:15:14.296975 *** vinum started ***
16 Feb 2004 20:15:14.323960 printconfig boot3.config
16 Feb 2004 20:37:55.783001 *** vinum started ***
16 Feb 2004 20:37:55.797851 list

Current output of "vinum list":

12 drives:
D eightgig              State: up	Device /dev/ad0h	Avail: 8056/8055 MB (100%)
D threegig              State: up	Device /dev/ad2h	Avail: 3093/3093 MB (100%)
D vinumdrive0           State: up	Device /dev/da0h	Avail: 4298/4298 MB (100%)
D vinumdrive1           State: up	Device /dev/da1h	Avail: 4298/4298 MB (100%)
D vinumdrive2           State: up	Device /dev/da2h	Avail: 4298/4298 MB (100%)
D vinumdrive3           State: up	Device /dev/da3h	Avail: 4298/4298 MB (100%)

6 volumes:
V root                  State: up	Plexes:       2	Size:        256 MB
V swap                  State: up	Plexes:       2	Size:        255 MB
V usr                   State: up	Plexes:       2	Size:       2048 MB
V var                   State: up	Plexes:       2	Size:        533 MB
V data                  State: up	Plexes:       1	Size:         16 GB
V junk                  State: up	Plexes:       1	Size:        664 MB

10 plexes:
P root.p0             C State: up	Subdisks:     1	Size:        256 MB
P swap.p0             C State: up	Subdisks:     1	Size:        255 MB
P usr.p0              C State: up	Subdisks:     1	Size:       2048 MB
P var.p0              C State: up	Subdisks:     1	Size:        533 MB
P root.p1             C State: up	Subdisks:     1	Size:        256 MB
P swap.p1             C State: up	Subdisks:     1	Size:        255 MB
P usr.p1              C State: up	Subdisks:     1	Size:       2048 MB
P var.p1              C State: up	Subdisks:     1	Size:        533 MB
P data.p0            R5 State: up	Subdisks:     5	Size:         16 GB
P junk.p0             C State: up	Subdisks:     1	Size:        664 MB

14 subdisks:
S root.p0.s0            State: up	PO:        0  B Size:        256 MB
S swap.p0.s0            State: up	PO:        0  B Size:        255 MB
S usr.p0.s0             State: up	PO:        0  B Size:       2048 MB
S var.p0.s0             State: up	PO:        0  B Size:        533 MB
S root.p1.s0            State: up	PO:        0  B Size:        256 MB
S swap.p1.s0            State: up	PO:        0  B Size:        255 MB
S usr.p1.s0             State: up	PO:        0  B Size:       2048 MB
S var.p1.s0             State: up	PO:        0  B Size:        533 MB
S data.p0.s0            State: up	PO:        0  B Size:       4296 MB
S data.p0.s1            State: up	PO:      492 kB Size:       4296 MB
S data.p0.s2            State: up	PO:      984 kB Size:       4296 MB
S data.p0.s3            State: up	PO:     1476 kB Size:       4296 MB
S data.p0.s4            State: up	PO:     1968 kB Size:       4295 MB
S junk.p0.s0            State: up	PO:        0  B Size:        664 MB 

Any assistance is appreciated,

Tony


More information about the freebsd-questions mailing list