Poor NFS server performance in 6.0 with SMP and mpsafenet=1

[Bernd Walter]

On Wed, Nov 02, 2005 at 02:58:36PM +0000, Gavin Atkinson wrote:
> Hi,
> 
> I'm seeing incredibly poor performance when serving files from an SMP
> FreeBSD 6.0RC1 server to a Solaris 10 client.  I've done some
> experimenting and have discovered that either removing SMP from the
> kernel, or setting debug.mpsafenet=0 in loader.conf massively improves
> the speed.  Switching preemption off seems to also help.
> 
> A quick summary of my findings:  switching SMP off and setting
> mpsafenet=0 gives more than 700% boost in performance.  PREEMPTION is a
> pessimisation in the SMP case, mpsafenet=1 is a pessimisation in all
> cases.  Given these are three of the major new features in 6.0, this
> doesn't seem ideal :)
> 
> The following are average times, in seconds, to serve a 512meg file over
> NFS to the Solaris host.  Each experiment was performed immediately
> after a reboot of the FreeBSD host, and after one untimed transfer (in
> order to warm the FreeBSD vfs cache).
> 
> No SMP, mpsafenet=1			 59.4
> No SMP, mpsafenet=0			 49.4
> No SMP, mpsafenet=1, no PREEMPTION	 53.1
> No SMP, mpsafenet=0, no PREEMPTION	 51.9
>    SMP, mpsafenet=1			351.7
>    SMP, mpsafenet=0			 74.5
>    SMP, mpsafenet=1, no PREEMPTION	264.9
>    SMP, mpsafenet=0, no PREEMPTION	 53.7

Which scheduler?

-- 
B.Walter                   BWCT                http://www.bwct.de
bernd at bwct.de                                  info at bwct.de