gstripe on 7B4 - was: gmirror on 7B4

Chris H. chris# at
Fri Jan 4 03:42:19 PST 2008

Quoting John Nielsen <john at>:

> Quoting "Chris H." <>:
>>> If you want specific advice for a specific scenario you can 
>>> probably get it, but you'll have to supply some additional details. 
>>> For instance I'm still not sure if this is a new install or an 
>>> upgrade
>> Both:
>> I was wondering why gmirror wasn't an option during sysinstall (the
>> creation, and installation to).
>> Which begged the question - now that it's installed...
>>> (even after re-reading the entire thread), or if da3 is the same 
>>> size as da0-2. Doing what you describe below will blow away the 
>>> existing contents of da3 and the other disks, and/or won't be 
>>> allowed if anything on da3 is currently mounted/running. Also you 
>>> should stop saying mirror if you mean stripe or JBOD. :)
>> Quite right. Again, my bad. I'm sorry this became so convoluted. It seemed
>> so clear at first. But as it started a question about gmirror, and my
>> almost immediate discovery that gmirror doesn't do RAID0, as I required.
>> Turned it into gstripe. I thought I had managed to make the transition
>> smoothly. But as you effectively indicated, no dice. Sorry. :(
>> Thank you *very* much for your informative, and thoughtful replies -
>> and patience. :)
>> OK, in the final analysis I've decided (now that it's (7B4) installed...)
>> I'll just keep /boot, /root (and presumably /dev) on the already available
>> and running install disk (da3).
>> Then perform:
>> # gstripe label -v -s 131072 bigstripe \
>> /dev/da0 /dev/da1 /dev/da2
>> # newfs -U /dev/stripe/bigstripe
>> # mkdir /bigstripe
>> # mount /dev/stripe/bigstripe /bigstripe
>> # echo 'geom_stripe_load="YES"' >> /boot/loader.conf
>> # echo '/dev/stripe/bigstripe /bigstripe ufs rw 2 2' >> /etc/fstab
> Good up to here. Now you still have your running system and existing 
> partitions on da3, and a new empty large raid0 volume mounted on 
> /bigstripe. Before continuing, you should ask yourself (and perhaps 
> tell the rest of us) what exactly do you want to use all of that 
> space for?

Sure. I simply want to utilize all the space available to me on the
2U rack system it's currently running on. It's local, so I have direct
access to it.

> da3 is probably large enough for the OS itself, and while
> it's not redundant at least you have better odds of not losing your 
> OS if a drive fails with this setup.

Well, there's kind of a funny story attached to this system. I got
up to stretch my legs, as I had been at the console for hours. So I
went for a fairly long walk. I happened to past a couple of dumpsters
during my journey and noticed a 2U rackmount system leaning against one
of them. I could see it had been there for at least a 'couple of days.
So now being confident that this was it's intended final resting place,
I picked it up and took it back with me. As I was sure that at least /some/
of the components were salvageable. After I got it back I popped the top
off of it, and had a look inside. I couldn't believe it. It had the same
Tyan board that most of the servers I manage here run. Both of the CPU
sockets were filled as well. So, I then took a careful look for any
visible damage, and couldn't find any, nor was I able to smell that
distinctive tell-tale oder that IC's, trace patterns, and such leave
after failure. So I looked it over one more time before plugging it in,
and discoverd that the single SIMM that was in it, was in the wrong
slot. So I moved it and took a chance, plugged it in and powered it
up. It POSTed as it should! Great! I just inherited a perfect system. :)
It had a copy of FreeBSD-4.3 on it. But it wouldn't boot (not that I
cared). So I did some more (deeper) probing, and discovered the 4
IBM SCSI drives installed in it had less than 20hrs. on them. They're
brand new! WOW, with that kind of luck, I should have bought a lottery
ticket. :)
To the point, given the time on these drives, and my experience with
IBM's line of ULTRA SCSI drives. I have little fear of failure for
quite some time. I also have a backup schedule that runs no less
than twice daily (immediately prior to, and immediately after build
world/kernel, install kernel/world as well). So this doesn't immediately
concern me. Many things are likely to change prior to my needing to
get concerned about drive failure.
As to keeping/wiping da3; I hadn't intended to wipe da3. But rather,
move the bulk of what's already there to the newly created stripe.
Then eventually (after ensuring everything works as intended) re-sizing
(shrinking) it down and adding the now available "slack" to the stripe

>> # cd /var
>> # tar cf - . | (cd /bigstripe; tar xvf -
>> and repeating the above two lines for
>> /bin, /compat, /dist, /entropy, /etc, /lib, /libexec,
>> /media, /mnt, /proc, /rescue, /sbin, /sys, /tmp, and /usr
> That will get your files moved, but what are you trying to accomplish here?

Move (mostly) everything off of da3.

>> moving and remaking /home. Then deleting and re-creating
>> the above (/bin, /compat, etc...).
> How do you propose to re-create them if they've been moved to a 
> different filesystem?

Given the two lines above (
# cd /var

# tar cf - . | (cd /bigstripe; tar xvf -)
I haven't actually /moved/ them. So, repeating them for all the other
folders I intend to re-locate, I will only have /duplicated/ them
on the newly created stripe pool. Had I added ; cd /; rm -rf /var
to the second of the two lines above, that would have been a /move/. :)
So by "re-create" I was meaning to remove the original folders and
"re-create" them so as to be mounted off of /

> At best you can create symlinks to them which will usually work, but 
> in this case I don't see a reason to go that route.

That's an interesting point to me. As I was a bit concerned about
symlinking them. This has caused me some greif in the past. Especially
where building from source is concerned. Seems some things don't follow
symlinks well. While I've managed to find and understand what I would
now consider /most/ of them. I don't want to /trip/ on any new ones. So
I had intended to mount the folders moved to "bigstripe" off root (/).
Have I overlooked something here?

>> Then modify /etc/fstab
>> to read /dev/stripe/bigstripe / ufs rw 2 2
> And this is the big question mark/red flag. If you get rid of da3 
> then you won't be able to boot,

Sorry if I mislead you on this. I hadn't intended to /remove/ da3.
Just move the bulk what's already on it to "bigstripe".

> and if you're keeping it anyway then why not use it? If you really 
> want to do this you should use dump/restore instead of tar above and 
> do the entire root filesystem (by which I mean "/" and not just 
> "/root"), then be careful to always update /boot and /etc on da3 any 
> time you update the system. Or in other words, you're asking for 
> trouble.

If I am understanding you correctly here, that's essentially what I'm
planning to do. I realize that I may not have described/articulated
my intention(s) as well as I might have. ;) Sorry.

>> unmount /bigstripe
> That should be umount,

Of course. :) I was thinking UNmount as I was writing. So I'm afraid
that's what my fingers typed. :P

> although you should probably just reboot with the new fstab if that's 
> what you really want.

Yes, after thinking about the whole thing for awhile. I decided it
made alot more sense to reboot after all the changes, and let the
system "do it's thing".

>> mount /
> Same as above.
>> Done. Yes?
> Err..
>> Maybe I'm overestimating the FreeBSD file system. But this
>> seems plausible.
> FreeBSD can handle it and you're definitely moving in the direction 
> of a workable setup here, but you may have gotten a bit carried away. 
> A better option might be to just move one mountpoint (such as /var) 
> over to the stripe volume by using dump/restore, then update fstab so 
> it gets mounted from the new location. If you want to move other 
> directories (such as /tmp or /home or even /usr) to the new volume 
> you can do so, you'll just need to create symlinks to their new 
> locations.

Symlink(s)? See my concerns with this above.

> If this is or was a new install you may want to start over and re-do 
> your partitioning with the end goal in mind (so you don't have unused 
> space or partitions on da3, for instance).

In the final analysis, this would /definitely/ seem the most /expedient/
approach, even on a long time system. So long as you have recent backups/
dumps to place in the newly created folders. I also decided that leaving:
/bin, /boot, /dev, /etc, /lib, /libexec, /proc, /rescue, /root, and /sbin
on da3 would probably be the best (safest) layout to go with also.

Thank you again for taking the time to share your thoughts on this. I
/really/ appreciate it.


>> Thanks to everyones time, consideration (and patience).
> Sure.
> JN
>>>> For the record, FSTAB (on da3):
>>>> /dev/da3s1b
>>>> none (swap)
>>>> /dev/da3s1a
>>>> /
>>>> /dev/da3s1d
>>>> /var
>>>> Thanks for your response.
>>>> Chris
>> A *little* history, perhaps helps context...
>> ---8<---snip---8<---
>>>>>>> OK, my mistake...
>>>>>>> Seems for my application (RAID0), *gstripe* is what I should
>>>>>>> be using.
>>>>>>> Q: But RAID0 provides 0 redundancy. How will you cope with data loss?
>>>>>>> A: Complete backups occur twice daily and I (we) use IP RAID0 -
>>>>>>> eg; 2 different servers have/provide the same data, and the DNS 
>>>>>>> provides
>>>>>>> "round-robin". Thereby spreading the requests roughly equal across
>>>>>>> both servers.
>>>>>>> So, given my new found knowledge. I felt I should probably ask before
>>>>>>> potentially clobbering (breaking) the server I'll be attempting 
>>>>>>> this on.
>>>>>>> Will the following accomplish my goal?
>>>>>>> Current setup:
>>>>>>> /dev indicates the following:
>>>>>>> da0, da0c, da0cs1, da0s1, da0s1c
>>>>>>> da1, da1c, da1cs1, da1s1, da1s1c
>>>>>>> da2, da2c, da2cs1, da2s1, da2s1c
>>>>>>> ...and the following, which FreeBSD is installed on:
>>>>>>> da3, da3s1, da3s1a, da3s1b, da3s1c, da3s1d
>>>>>>> All drives are of same size/make/model.
>>>>>>> Given the above, I intend to issue the following:
>>>>>>> # gstripe label -v -s 131072 bigstripe \
>>>>>>> /dev/da0 /dev/da1 /dev/da2 /dev/da3
>>>>>>> # newfs -U /dev/stripe/bigstripe
>>>>>>> # mount /dev/stripe/bigstripe /bigstripe
>>>>>>> # echo 'geom_stripe_load="YES"' >> /boot/loader.conf
>>>>>>> # echo '/dev/stripe/bigstripe /bigstripe ufs rw 2 2' >> /etc/fstab
>>>>>> Yes, this should be fine (though you may need to do a "gstripe 
>>>>>> load" near the beginning).
>>>>>>> Or do/should I issue:
>>>>>>> # gconcat label -v extradisks /dev/da0 /dev/da1 /dev/da2
>>>>>>> # gstripe label -v bigstripe /dev/da3 /dev/concat/extradisks
>>>>>>> # bsdlabel -wB /dev/stripe/bigstripe
>>>>>>> # newfs -U /dev/stripe/bigstripe
>>>>>>> # mount /dev/stripe/bigstripe /bigstripe
>>>>>> No, assuming the disks are (roughly) the same size there's no 
>>>>>> reason to use gconcat, and in this case doing so will likely 
>>>>>> hurt performance in addition to adding complexity. gconcat is 
>>>>>> generally just for JBOD-type scenarios and it sounds like you're 
>>>>>> after RAID0 which is what gstripe is for.
>>>>>> JN
>> _______________________________________________
>> freebsd-stable at mailing list
>> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at"
> _______________________________________________
> freebsd-stable at mailing list
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at"

panic: kernel trap (ignored)

More information about the freebsd-stable mailing list