One Last Plea For Vinum Assistance
Drew Tomlinson
drew at mykitchentable.net
Thu Jan 20 13:19:10 PST 2005
Thanks, for your reply. I at least *feel* better knowing that my
questions doesn't fall into the "too dumb to deserve an answer"
category. :)
On 1/19/2005 10:57 PM Ted Mittelstaedt wrote:
>Hi Drew,
>
> Please read the following:
>
>http://www.vinumvm.org/vinum/how-to-debug.html
>
>
I have.
> And follow the instructions exactly. And I mean exactly.
>
>
I thought I had, with the exception of the /var/log/vinum_history and
/var/log/messages files due to the size. However I offered to provide
them separately.
>Also keep the following in mind, Greg will try to help but
>note carefully the sentence on this webpage:
>
>"Since I wrote it, FreeBSD has changed its I/O structure, breaking many
>things in Vinum. At the time of writing, a new version, provisionally
>called gvinum, is being written"
>
>
Yes, however I thought that vinum is still the way to go with 4-STABLE
versions. Is this still true?
> I myself have had one serious crash on a vinum RAID volume as
>a result of a SCSI cable problem that blew away the volume. (2
>drives were corrupted, instead of just one, making it impossible
>for the volume manager to repair by itself) I sent all the info
>to Greg but ultimately he wasn't able to offer any suggestions
>on recovering the array so I just wiped it and started over.
>
>Note that Greg DID NOT recommend wiping the array. In fact he
>didn't recommend anything. The lack of any recommendation
>appears to be his way of telling you "your volume is screwed,
>wipe it and start over" Like most UNIX commands, if Greg
>has nothing to offer, he says nothing at all, he won't tell you he
>has nothing to offer. So, the lack of a response to your
>original post you can probably take as an answer, to be honest.
>
>
I can understand that about Greg. I'm sure he has so many emails to
deal with each day that there's no point spending time on replies that
don't offer any benefit. But what really blows me away is that my
volumes work if they are deleted and re-created. This tells me my
volume is not "screwed" because they work. I just don't understand why
they don't survive reboots. These are the same volumes that I've had
since version 4.1 and they've worked flawlessly -- well there was the
time I tried to add a firewire drive to a concatenated volume but had
the SCSI delay set too low in the kernel so the drive was not available
when vinum started. However once I figured that out (with help from
Hidetoshi Shimokawa, the driver author), everything worked fine again.
However the upgrade from 4.9 to 4.10 royally screwed up the vinum
volumes and they haven't been right since.
> This did teach me a lesson that I kind of knew already but
>didn't think too much about. That is, a software array is no substitute
>for a hardware array. In other words, vinum is a great thing
>if what your wanting to do is use a bunch of cheap disks and
>cheap controller cards to either get a giant partition, or to
>stripe them together and get faster access. But it's not so
>good if the intent is to get some crash recovery.
>
>
Agreed. I have backups for crash recovery. With the exception of
/usr, I used vinum so that I could have large drives on which to store
backups, mp3s, mpegs, online copies of software, etc.
> I don't use and have never used vinum for /etc, /, /usr, /var
>or any other system partitions. I only use it for partitions
>that I want to mount AFTER the system is booted. If I were in
>your shoes I'd nuke your system and start all over again and
>rethink how I had it laid out. I would use a single disk for
>the system then take the rest of the disks and put them together
>under vinum. Then I'd mount that on /ftp and I'd softlink
>whatever thing is gopping up space under /usr, for example
>/usr/local/www, to a directory under /ftp
>
>
You have a valid point and I will heed your advice when I finally wipe
and upgrade to 5-STABLE. In the beginning, it seemed to be a good idea
to use vinum to stripe my /usr for speed. Since this is just a
home/hobby system, I probably never really needed the speed increase but
I wanted to experiment.
>Vinum isn't going to give you any crash recovery
>for /usr so there is really no point in making /usr a vinum
>volume. Beyond that I really don't understand why you
>are putting /usr as a vinum volume, espically as you yourself
>said "Fortunately this volume is up and running or I would
>really be in a mess" I mean, your basically saying your
>hitting yourself in the face and you feel fortunate you
>haven't broken your nose yet.
>
>
You're right and my nose is still straight. :) See above as to "why" I
striped /usr.
>Anyway, one other thing I will bring up: How exactly did
>you update your system? Did you nuke and repave it? Or did
>you follow the instructions here EXACTLY:
>
>http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html
>
>
As I have since version 4.1. My steps are:
make buildworld
make buildkernel
make installkernel
make installworld
mergemaster (accept replacement of all files I have not modified and
merge my mods with new files)
reboot
I see 19.4.1 has two steps between "make installkernel" and "make
installworld" that I did not follow. Do you suspect this is where my
problem lies? I need to upgrade to the latest security level so I could
try this.
>If you didn't do one or the other of these things then nobody is going
>to help you.
>
>Ted
>
>
Thanks your your time and input!
Drew
>>-----Original Message-----
>>From: owner-freebsd-questions at freebsd.org
>>[mailto:owner-freebsd-questions at freebsd.org]On Behalf Of Drew Tomlinson
>>Sent: Tuesday, January 18, 2005 2:00 PM
>>To: FreeBSD Questions
>>Subject: One Last Plea For Vinum Assistance
>>
>>
>>I sent the message below a couple of times but did not receive
>>any response.
>>I assume that it's because either I have a really difficult
>>problem or am
>>asking something really stupid. :) But anyway, I want to install
>>additional
>>memory in this machine and am sure I will run across the same problems
>>after
>>shutting down. So if anyone has any suggestions on how I
>>might solve this
>>issue, I'd really appreciate the input.
>>
>>Thanks,
>>
>>Drew
>>
>>--- Original Message ---
>>Since an upgrade from 4.9 to 4.10, I've had problems with
>>vinum. The basic
>>problem is that upon reboot, two of my vinum drives show up as
>>"referenced" and
>>thus create the associated chaos. I've tried many things and fiddled
>>around
>>quite a bit so I can't say exactly what I've done. I can
>>include all of
>>the
>>entries in the history file since Oct. 31 if that's a help but
>>it would
>>be a
>>long list.
>>
>>So prior to digging that deep, I will describe where I stand
>>currently and
>>where I want to finish. Currently, I have one vinum volume
>>that I use for
>>/usr. Fortunately this volume is up and running or I would
>>really be in a
>>mess. Here's the 'vinum list' output in this state:
>>
>>blacklamb# vinum
>>vinum -> list
>>2 drives:
>>D disk1 State: up Device /dev/da0s1h Avail:
>>0/8383 MB (0%)
>>D disk2 State: up Device /dev/da1s1h Avail:
>>0/8383 MB (0%)
>>
>>1 volumes:
>>V usr State: up Plexes: 1 Size:
>> 16 GB
>>
>>1 plexes:
>>P usr.p0 S State: up Subdisks: 2 Size:
>> 16 GB
>>
>>2 subdisks:
>>S usr.p0.s0 State: up PO: 0 B Size:
>> 8383 MB
>>S usr.p0.s1 State: up PO: 256 kB Size:
>> 8383 MB
>>
>>I want to add another volume and mount it on /ftp. After creating the
>>volume,
>>vinum sees it and it appears OK as indicated in this output:
>>
>>vinum -> list
>>5 drives:
>>D disk1 State: up Device /dev/da0s1h Avail:
>>0/8383 MB (0%)
>>D disk2 State: up Device /dev/da1s1h Avail:
>>0/8383 MB (0%)
>>D ftp1 State: up Device /dev/ad0s1h Avail:
>>0/76319 MB (0%)
>>D ftp2 State: up Device /dev/ad1s1h Avail:
>>0/76319 MB (0%)
>>D ftp3 State: up Device /dev/da3s1h Avail:
>>0/114473 MB (0%)
>>
>>2 volumes:
>>V usr State: up Plexes: 1 Size:
>> 16 GB
>>V ftp State: up Plexes: 1 Size:
>> 260 GB
>>
>>2 plexes:
>>P usr.p0 S State: up Subdisks: 2 Size:
>> 16 GB
>>P ftp.p0 C State: up Subdisks: 3 Size:
>> 260 GB
>>
>>5 subdisks:
>>S usr.p0.s0 State: up PO: 0 B Size:
>> 8383 MB
>>S usr.p0.s1 State: up PO: 256 kB Size:
>> 8383 MB
>>S ftp.p0.s0 State: up PO: 0 B Size:
>> 74 GB
>>S ftp.p0.s1 State: up PO: 74 GB Size:
>> 74 GB
>>S ftp.p0.s2 State: up PO: 149 GB Size:
>> 111 GB
>>
>>Next I 'quit' the vinum command line and fsck my newly created
>>volume. It
>>finishes successfully and I mount it:
>>
>>blacklamb# df
>>Filesystem 1K-blocks Used Avail Capacity Mounted on
>>/dev/da0s1a 302350 102088 176074 37% /
>>/dev/vinum/usr 16639674 5433762 9874739 35% /usr
>>procfs 4 4 0 100% /proc
>>/dev/vinum/ftp 265119539 119729707 132133856 48% /ftp
>>
>>I recheck with the 'vinum list' command and the output is the same as
>>above.
>>Now I cross my fingers and reboot to see if the volume comes
>>up properly on
>>startup. However ftp3 (/dev/da3s1h) comes up 'referenced' and causes
>>problems.
>>
>>Mounting root from ufs:/dev/da0s1a
>>dumpon: crash dumps to /dev/da1s1b (13, 131081)
>>vinum: loaded
>>vinum: reading configuration from /dev/ad1s1h
>>vinum: ftp.p0.s2 is crashed
>>vinum: ftp.p0 is corrupt
>>vinum: updating configuration from /dev/ad0s1h
>>vinum: updating configuration from /dev/da1s1h
>>vinum: updating configuration from /dev/da0s1h
>>vinum: /dev is mounted read-only, not rebuilding /dev/vinum
>>Warning: defective objects
>>
>>D ftp3 State: referenced Device Avail: 0/0 MB
>>P ftp.p0 C State: corrupt Subdisks: 3 Size:
>> 260 GB
>>S ftp.p0.s2 State: crashed PO: 149 GB Size:
>> 111 GB
>>swapon: adding /dev/da1s1b as swap device
>>Automatic boot in progress...
>>/dev/da0s1a: FILESYSTEM CLEAN; SKIPPING CHECKS
>>/dev/da0s1a: clean, 100130 free (938 frags, 12399 blocks, 0.6%
>>fragmentation)
>>/dev/vinum/ftp: CANNOT READ: BLK 546979872
>>/dev/vinum/ftp: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
>>/dev/vinum/ftp: CANNOT WRITE: BLK 2800
>>/dev/vinum/ftp: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
>>/dev/vinum/usr: FILESYSTEM CLEAN; SKIPPING CHECKS
>>/dev/vinum/usr: clean, 11205903 free (152463 frags, 1381680
>>blocks, 0.9%
>>fragmentation)
>>THE FOLLOWING FILE SYSTEM HAD AN UNEXPECTED INCONSISTENCY:
>> /dev/vinum/ftp (/ftp)
>>File system preen failed, trying fsck -y . . .
>>** /dev/da0s1a
>>** Last Mounted on /
>>** Root file system
>>** Phase 1 - Check Blocks and Sizes
>>** Phase 2 - Check Pathnames
>>** Phase 3 - Check Connectivity
>>** Phase 4 - Check Reference Counts
>>** Phase 5 - Check Cyl groups
>>2053 files, 51045 used, 100130 free (938 frags, 12399 blocks, 0.6%
>>fragmentation)
>>** /dev/vinum/usr
>>** Last Mounted on /usr
>>** Phase 1 - Check Blocks and Sizes
>>** Phase 2 - Check Pathnames
>>** Phase 3 - Check Connectivity
>>** Phase 4 - Check Reference Counts
>>** Phase 5 - Check Cyl groups
>>401072 files, 5433771 used, 11205903 free (152463 frags,
>>1381680 blocks,
>>0.9% fragmentation)
>>** /dev/vinum/ftp
>>
>>CANNOT READ: BLK 546979872
>>UNEXPECTED SOFT UPDATE INCONSISTENCY
>>
>>CONTINUE? yes
>>
>>THE FOLLOWING DISK SECTORS COULD NOT BE READ: 546979872, 546979873,
>>546979874, 546979875,
>>/dev/vinum/ftp: CANNOT FIGURE OUT FILE SYSTEM PARTITION
>>
>>UNEXPECTED SOFT UPDATE INCONSISTENCY
>>WARNING: R/W mount of /ftp denied. Filesystem is not clean - run fsck
>>mount: /dev/vinum/ftp: Operation not permitted
>>Mounting /etc/fstab filesystems failed, startup aborted
>>Enter full pathname of shell or RETURN for /bin/sh:
>>
>>I enter the shell and run 'vinum list':
>>
>>vinum -> list
>>4 drives:
>>D ftp1 State: up Device /dev/ad0s1h Avail:
>>0/76319 MB (0%)
>>D ftp2 State: up Device /dev/ad1s1h Avail:
>>0/76319 MB (0%)
>>D disk1 State: up Device /dev/da0s1h Avail:
>>0/8383 MB (0%)
>>D disk2 State: up Device /dev/da1s1h Avail:
>>0/8383 MB (0%)
>>D ftp3 State: referenced Device Avail: 0/0 MB
>>
>>2 volumes:
>>V usr State: up Plexes: 1 Size:
>> 16 GB
>>V ftp State: up Plexes: 1 Size:
>> 260 GB
>>
>>2 plexes:
>>P usr.p0 S State: up Subdisks: 2 Size:
>> 16 GB
>>P ftp.p0 C State: corrupt Subdisks: 3 Size:
>> 260 GB
>>
>>5 subdisks:
>>S usr.p0.s0 State: up PO: 0 B Size:
>> 8383 MB
>>S usr.p0.s1 State: up PO: 256 kB Size:
>> 8383 MB
>>S ftp.p0.s0 State: up PO: 0 B Size:
>> 74 GB
>>S ftp.p0.s1 State: up PO: 74 GB Size:
>> 74 GB
>>S ftp.p0.s2 State: crashed PO: 149 GB Size:
>> 111 GB
>>
>>Now I delete all references to the ftp volume by running these
>>commands:
>>
>>vinum -> rm -f ftp.p0.s2
>>vinum: removing ftp.p0.s2
>>Correcting length of ftp.p0: was 547043829, is 312602446
>>vinum: ftp.p0 is up
>>vinum -> rm -f ftp.p0.s1
>>vinum: removing ftp.p0.s1
>>Correcting length of ftp.p0: was 312602446, is 156301223
>>vinum -> rm -f ftp.p0.s0
>>vinum: removing ftp.p0.s0
>>vinum -> rm -f ftp.p0 vinum: removing ftp.p0
>>vinum: ftp is down
>>vinum -> rm -f ftp vinum: removing ftp
>>vinum -> rm -f ftp3
>>vinum -> rm -f ftp2
>>vinum -> rm -f ftp1
>>
>>Yet the 'reference to ftp3 remains.
>>
>>vinum -> list
>>1 drives:
>>D disk1 State: up Device /dev/da0s1h Avail:
>>0/8383 MB (0%)
>>D disk2 State: up Device /dev/da1s1h Avail:
>>0/8383 MB (0%)
>>D ftp3 State: referenced Device Avail: 0/0 MB
>>
>>1 volumes:
>>V usr State: up Plexes: 1 Size:
>> 16 GB
>>
>>1 plexes:
>>P usr.p0 S State: up Subdisks: 2 Size:
>> 16 GB
>>
>>2 subdisks:
>>S usr.p0.s0 State: up PO: 0 B Size:
>> 8383 MB
>>S usr.p0.s1 State: up PO: 256 kB Size:
>> 8383 MB
>>
>>Why does the list output show "1 drives:" when there are three listed
>>and two
>>working? Anyway, I've been here before and I quit vinum.
>>Then I issue
>>these
>>commands:
>>
>># df
>>Filesystem 512-blocks Used Avail Capacity Mounted on
>>/dev/da0s1a 604700 204180 352144 37% /
>>/dev/vinum/usr 33279348 10867544 19749458 35% /usr
>>procfs 8 8 0 100% /proc
>># umount /usr
>># vinum stop
>>vinum: unloaded
>>vinum unloaded
>># vinum start
>>vinum: loaded
>>vinum: reading configuration from /dev/da1s1h
>>vinum: updating configuration from /dev/da0s1h
>>
>>Then I issue a 'vinum list' and all seems well.
>>
>># vinum
>>vinum -> list
>>2 drives:
>>D disk1 State: up Device /dev/da0s1h Avail:
>>0/8383 MB (0%)
>>D disk2 State: up Device /dev/da1s1h Avail:
>>0/8383 MB (0%)
>>
>>1 volumes:
>>V usr State: up Plexes: 1 Size:
>> 16 GB
>>
>>1 plexes:
>>P usr.p0 S State: up Subdisks: 2 Size:
>> 16 GB
>>
>>2 subdisks:
>>S usr.p0.s0 State: up PO: 0 B Size:
>> 8383 MB
>>S usr.p0.s1 State: up PO: 256 kB Size:
>> 8383 MB
>>vinum ->
>>
>>Since everything appears correct, this seems like a good place
>>to do 'vinum
>>saveconfig'. Next I edit /etc/fstab so only /ftp is not mounted and
>>reboot.
>>The system reboots fine and the 'vinum list' is as it is
>>immediately above.
>>
>>
>>Running in full production mode, I create the ftp volume again:
>>
>>vinum -> create -f vinum_ftp.conf
>>Dec 24 16:37:45 blacklamb /kernel: vinum: drive ftp1 is up
>>Dec 24 16:37:45 blacklamb /kernel: vinum: drive ftp2 is up
>>Dec 24 16:37:45 blacklamb /kernel: vinum: drive ftp3 is up
>>5 drives:
>>D disk1 State: up Device /dev/da0s1h Avail:
>>0/8383 MB (0%)
>>D disk2 State: up Device /dev/da1s1h Avail:
>>0/8383 MB (0%)
>>D ftp1 State: up Device /dev/ad0s1h Avail:
>>0/76319 MB (0%)
>>D ftp2 State: up Device /dev/ad1s1h Avail:
>>0/76319 MB (0%)
>>D ftp3 State: up Device /dev/da3s1h Avail:
>>0/114473 MB (0%)
>>
>>2 volumes:
>>V usr State: up Plexes: 1 Size:
>> 16 GB
>>V ftp State: up Plexes: 1 Size:
>> 260 GB
>>
>>2 plexes:
>>P usr.p0 S State: up Subdisks: 2 Size:
>> 16 GB
>>P ftp.p0 C State: up Subdisks: 3 Size:
>> 260 GB
>>
>>5 subdisks:
>>S usr.p0.s0 State: up PO: 0 B Size:
>> 8383 MB
>>S usr.p0.s1 State: up PO: 256 kB Size:
>> 8383 MB
>>S ftp.p0.s0 State: up PO: 0 B Size:
>> 74 GB
>>S ftp.p0.s1 State: up PO: 74 GB Size:
>> 74 GB
>>S ftp.p0.s2 State: up PO: 149 GB Size:
>> 111 GB
>>Dec 24 16:37:45 blacklamb /kernel: vinum: ftp.p0.s0 is up
>>Dec 24 16:37:45 blacklamb /kernel: vinum: ftp.p0.s1 is up
>>Dec 24 16:37:45 blacklamb /kernel: vinum: ftp.p0.s2 is up
>>Dec 24 16:37:45 blacklamb /kernel: vinum: ftp.p0 is up
>>Dec 24 16:37:45 blacklamb /kernel: vinum: ftp is up
>>
>>After running fsck, I can mount and access the volume. The system
>>continues to
>>run fine until the next reboot where the same problems start all over
>>again. I
>>also have the same problem with another volume I wish to run called
>>"backup".
>>How can I fix my vinum volumes so they survive reboots?
>>
>>Thanks for your help and Happy Holidays!
>>
>>Drew
>>
>>
More information about the freebsd-questions
mailing list