Significant performance regression for increased maxsockbuf on 8.0-RELEASE

[Steven Hartland]

As noted in the FreeBSD TCP tuning and performance  thread here:
http://lists.freebsd.org/pipermail/freebsd-performance/2009-December/003909.html

There seems to be a significant performance drop when using 8.0 vs 7.0
after digging around this seems to be caused by the use of increased
kern.ipc.maxsockbuf

Using the default value ( 262144 ) I see the following iperf results:
iperf -t 10 -c server1 -w 2.5M -l 2.5M
------------------------------------------------------------
Client connecting to db3, TCP port 5001
TCP window size: 65.0 KByte (WARNING: requested 2.50 MByte)
------------------------------------------------------------
[  3] local 85.236.96.66 port 50848 connected with 85.236.96.42 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.09 GBytes    935 Mbits/sec


With anything larger than that the rate drops off significantly e.g. 282144
gives:-
iperf -t 10 -c db3 -w 2.5M -l 2.5M
------------------------------------------------------------
Client connecting to db3, TCP port 5001
TCP window size: 65.0 KByte (WARNING: requested 2.50 MByte)
------------------------------------------------------------
[  3] local 85.236.96.66 port 25867 connected with 85.236.96.42 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec    778 MBytes    651 Mbits/sec



and 302144 we're down to just under half the baseline / line rate
iperf -t 10 -c db3 -w 2.5M -l 2.5M
------------------------------------------------------------
Client connecting to db3, TCP port 5001
TCP window size: 65.0 KByte (WARNING: requested 2.50 MByte)
------------------------------------------------------------
[  3] local 85.236.96.66 port 12506 connected with 85.236.96.42 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec    588 MBytes    492 Mbits/sec

Any ideas why using window sizes above 65K results in such poor performance
on 8.0?

    Regards
    Steve


================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 

In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
or return the E.mail to postmaster at multiplay.co.uk.