mfs and buildworlds on da SunFire x4600

[Mars G. Miro]

On 5/3/07, Kris Kennaway <kris at obsecurity.org> wrote:
> On Thu, May 03, 2007 at 02:39:55AM +0800, Mars G. Miro wrote:
> > Greetz,
> >
> >    I got my hands on a SunFire x4600. This is a dual-core 8-way
> > Opteron (dmesg shows 16 CPUs) w/c has 32G RAM. FreeBSD 6.2/amd64 runs
> > fine on it despite:
> > - 'reboot' doesnt work, machine just hangs in there.
>
> There is a sysctl to specify an alternate reboot method.
>
> > - front USB ports wont work for a USB keyboard, just use da ports at da
> > back.
> >
> >   I've read a lot of threads in da past (and da BSDCan 2006 paper)
> > regarding mfs on /usr/src and /usr/obj to speed up buildworlds,
> > unfortunately it doesnt seem to improve things. I've tried both a
> > malloc and swap based async non-softupdates mfs (and rebooted da box
> > every after) but my buildworld speeds seem to be just da same as I do
> > it on da MPT SAS disk w/c is just around ~18mins.
>
> It is probably a combination of issues:
>
> 1) There is still some disk access because of accessing things like
> binaries in /usr.  This will hurt more for parallel builds because
> your disk hardware is single-threaded, so I/O delays will serialize
> you.  If you really wanted you could compare to a build in a md-hosted
> chroot to completely eliminate the disk, but this is just a game.
>
> 2) Buildworld doesn't parallelize well to 16 CPUs.  Large parts of it
> are completely non-parallel.  This is probably your main problem.
>

I guess that might explain it, see more below...

> You definitely should be measuring some difference - you need to use
> time(1) though instead of just estimating.
>

I did use time, here:

make -j32 buildworld:
disk
3229.975u 8981.269s 18:55.21 1075.6%    3804+2164k 19702+6594io 2158pf+0w

malloc mfs ( mdmfs -SM -o async,noatime -s 2G md {src, obj} )
3069.783u 9546.250s 18:57.58 1109.0%    3844+2147k 18504+4422io 2628pf+0w

swap mfs ( mdmfs -S -o async,noatime -s 2G md {src, obj} )
3061.682u 9682.776s 19:08.90 1109.2%    3849+2164k 13732+4402io 2053pf+0w


make -j16 buildworld:
disk
3218.722u 7855.885s 17:42.26 1042.5%    3833+2267k 16995+6620io 2445pf+0w

malloc mfs ( mdmfs -SM -o async,noatime -s 2G md {src, obj} )
3187.753u 8199.049s 17:57.63 1056.6%    3839+2238k 6890+4436io 2346pf+0w

swap mfs ( mdmfs -S -o async,noatime -s 2G md {src, obj} )
3200.400u 8213.312s 18:06.88 1050.1%    3860+2243k 12800+4397io 2303pf+0w


> Kris
>

Thanks


cheers
mars