Building RELENG_9 (or RELENG_9_*) on a small machine?

[Ian Smith]

On Fri, 1 Mar 2013 11:36:58 -0800, Darren Pilgrim wrote:
 > On 2013-03-01 10:50, Ian Smith wrote:
 > > At 256MB - the
 > > minimum earlier that completed installation without disabling CTL - swap
 > > often sat at ~14MB but blew out to around 165MB building those huge llvm
 > > libraries - cc1plus 332M, 173M resident was one top I snipped, but I
 > > can't say I caught the biggest.
 > 
 > I had top running throughout a build and saw cc1plus reach 460 MB with 171MB
 > resident, at that point CPU was down to about 4% and the system was hammering
 > swap.  As a testament to FreeBSD's robustness, I could still log in via ssh,
 > start screened shells, and generally conduct admin tasks while cc1plus beat
 > the crap out of my VM.  Even start and stop other services.

After 15 years I expect nothing less; even so I try not to fill swap :)

 > > Then I added 128MB (to 384MB) and repeated the first buildworld (incl.
 > > clang) expecting huge savings as it'd only touched swap to about 12MB a
 > > time or two, mostly having 100MB+ of free memory .. wow, down to 7h02m!

Well I was going to bring mine straight up to 768MB, but having a spare 
512MB stick and in the interest of science, I tried again with 512MB and 
nothing in src.conf, but still CAM CTL disabled.  After a couple of 
hours swap was tickled up to 608K and got no higher, finishing at 544K 
which I suspect was VM getting a bit nervous and making plans at most; 
still at 544K next day.

As mentioned but more precisely, with 256MB buildworld took 7:39:38 with 
lots of swapping - but only 3:31:49 WITHOUT_CLANG=y, with no swapping - 
and then with 384MB, 7:02:10 and no more than 14MB swap seen.

With 512MB and sub-1MB swap touched it took 7:00:49; the extra 128MB RAM 
shaved a whole minute 31 seconds off buildworld, 0.15% faster, whoopee!

 > For 9.x, I changed my notes to "256 MB to run, 768 MB to build".  For 8.x,
 > the numbers were 192 MB to run, 512 MB to build.

Without CAM CTL I just managed to install 9.1 in 128MB; with took 256MB. 
If adrian@ succeeds defaulting CAM CTL off it'll improve matters ~35MB.

>From the above I'd say 384MB without CAM CTL or 512MB with will build 
world with minimal swapping, and 768MB will be fine - though I don't 
know about building bigger ports in that?  I'm not moving my $wholelife 
to 9.1 till packages become again available, for obvious reasons.

Any idea what happens both time- and memorywise from building world 
with clang rather than gcc?  Hopefully All That Code is more efficient?

 > > Here at least, building llvm libs and clang doubles buildworld time! and
 > > extends /usr/obj from 675MB to 1GB.
 > 
 > I'll be doing `make buildkernel buildworld` ET and size comparisons between
 > RELENG_8_3 and RELENG_9_1 when I test out my buildbox.  I'd like to gather
 > memory usage metrics as well, if someone knows some tricks for that.  My
 > current approach is somewhat crude. :)  If there's interest I'll follow up
 > with the results here.

Interest here.  My sampling was very crude also, just appending: 'nice 
top -nS -ores | head 11 | grep -v "^$" ; sleep 60' to a log.  Doesn't 
catch max size or res except by chance, but quite good enough to track 
movements in swap.  I guess parsing the tail of vmstat might be the go?

cheers, Ian