FreeBSD 5.x performance tips (ISC)

[Andre Guibert de Bruet]

On Mon, 12 Jan 2004, Peter Losher wrote:

> So, as many of you know ISC hosts a quad-Xeon server running FreeBSD 5.1
>  (-p10 to be precise) which hosts half of ftp.freebsd.org, etc.  Many of you
> helped out with some teething pains w/ virtual memory sizes, and kernel
> panics.  Thanks :)
>
> The issue with the system now is that while the kernel is SMP-aware, and as I
> watch 5.2-REL get downloaded today, this system is like the arm muscle that
> is developed to lift that barbell, but not enough blood is getting
> everywhere, so the barbell is slowly moving up while the muscle cramps like
> hell.  In this case the system is ~70% idle, and around 150 processes are
> locked and the performance starts to seriously decrease at times. (Entropy
> stops getting collected, etc.)  Not a pretty sight.  The CPU's are all
> spinlocking on an I/O channel. so high I/O translates into artificial high
> cpu and load averages.

What state does top report these processes in? On a busy ftp server, I
would expect *Giant, getblk, biord or select. What kind of load averages
are you seeing under load, and when somewhat idle? If this system is
currently contending on Giant, 5.2 will still exhibit this behavior but to
a lesser degree.

> So where can I look for pointers on how I can squeeze better performance out
> of this configuration? I already have the usual sysctl entries installed.
> Any chance moving to 5.2 will help the situation?

We're presently running one of the 5.2RC releases on a couple of
mailservers at work. It has fixed the load issues that we had seen related
to mixed I/O on our amr controllers with 5.1. YMMV.

For a little performance boost, try the ZERO_COPY_SOCKETS and
ADAPTIVE_MUTEXES combo in your kernel config. It Works Here (TM). :)

Hope this helps,

> Andre Guibert de Bruet | Enterprise Software Consultant >
> Silicon Landmark, LLC. | http://siliconlandmark.com/    >