scripting the buildworld/installworld process
kstewart at owt.com
Sun Aug 31 17:45:30 PDT 2003
On Sunday 31 August 2003 04:05 pm, Charles Howse wrote:
> > bw w -j5
> > 893.563u 311.353s 30:31.44 65.7% 1302+1591k 53927+138930io
> > 2331pf+0w
> > The wall clock time is really how efficient your process is. The
> > fact that your user time or sys time is faster doesn't mean much if
> > the wall
> > clock time is 1/3 longer.
> Wall clock? Would that be 30:31.44 from above?
> > I think that once you have it setup, I will probably make my
> > KISS simple
> > script more complicated :). I have one machine that the mouse
> > doesn't
> Glad to share, if that's what you mean.
That is what I meant. You learn by trying things.
> After re-reading your page about scripting the buildworld, I see an
> opportunity to test my script.
> I never thought about putting /usr/src and /usr/obj on different
> HDD's. I have 2 ATAPI HDD in my box, on different IDE busses, so that
> should work out well.
> I'll also need to adjust some slice sizes as well.
> How much space do you recommend for /var if I'm logging like you do?
> It's currently:
> Disk1 -
> / 100M
> swap 256M
> /var 200M
> /usr <remainder> about 1.8G
> Disk2 -
> /disk2 <entire disk> 7.8G
My smallest HD is 13GB and the system had 4 of them. My typical system
is on a 3x30GB ATA-100/133 HDs. Since, I have a lot of free space, my
partitions are typically 1.5GB. FreeBSD 4.x creates an ../obj of around
300-400MB; however, FreeBSD 5.x produces a 700-800MB.
I like to mount a partition as /usr/obj and /usr/src. The builds work
just fine with a link to /disk2/obj as /usr/obj. The reason for
separate partition is that I have messed up my system to the point that
I had to nfs_mount /usr/src and /usr/obj from a different machine. If I
mount /disk2/obj as /usr/obj, the make installworld dies because it
knows the true path to the obj files.
If I redo a system, I create a mount point as /usr/obj. Until then, I
just link /usr/obj to /usr2/obj and etc. I think that just moving
/usr/obj to your 2nd HD as a link would separate the I/O.
For FreeBSD 5.x, you need a lot more space in /. When I installed
5.0-release, I created / as 500MB and it is about 24% used. If you have
to manually download some tarballs, you can chew up space in a hurry.
The files in /var/log/build can fill up a fs really fast because each
system update is around 7-8MB. Since I log everything from cvsup to
"make index", I fill up space in a hurry :). If something breaks, I
want to be able to point to the commit that broke it.
> So I assume /usr/obj should be on the second HDD?
> About how much space should I reserve?
> Looks like I should also adjust my script to remove the -j4, eh?
Like you said, play with it. I use write caching and softupdates. They
all made it faster. Depending on your system, you may find that -j is
faster. From my experience, all of my systems (6 of various speeds and
cpus), do a build world faster without a -j.
When I was doing benchmarks on a Cray X/MP with 100MB/s & 1000MB/s data
channels to stripped disks capable of 20MB/s continuous, write caching
doubled the throughput of the system. The gain was not obvious up front
and everyone was surprised by the result. After we saw it, it was a
shoulder shrug because it appeared obvious that you want a process to
have faster access to data that it needs than forcing it to save
processed data first. In the commercial world, protecting the processed
data has a much higher assigned value. I went for speed. If someone
said 4-stable was broken, I wanted to be able to test the builds in
less than 30 minutes. I also didn't want to spend a lot of money to be
able to do that :).
BTW, those build times all had setiathome running in the background.
Having it running in the background really doesn't change the elapsed
time too much exept in 5.x. I kill seti before I do a timed run on it.
More information about the freebsd-questions