8.0-RC3 network performance regression
Robert Watson
rwatson at FreeBSD.org
Thu Nov 19 09:11:20 UTC 2009
On Wed, 18 Nov 2009, Elliot Finley wrote:
> I have several boxes running 8.0-RC3 with pretty dismal network performance.
> I also have some 7.2 boxes with great performance. Using iperf I did some
> tests:
>
> server(8.0) <- client (8.0) == 420Mbps
> server(7.2) <- client (7.2) == 950Mbps
> server(7.2) <- client (8.0) == 920Mbps
> server(8.0) <- client (7.2) == 420Mbps
>
> so when the server is 7.2, I have good performance regardless of whether the
> client is 8.0 or 7.2. when the server is 8.0, I have poor performance
> regardless of whether the client is 8.0 or 7.2.
>
> Has anyone else noticed this? Am I missing something simple?
I've generally not measured regressions along these lines, but TCP performance
can be quite sensitive to specific driver version and hardware configuration.
So far, I've generally measured significant TCP scalability improvements in 8,
and moderate raw TCP performance improvements over real interfaces. On the
other hand, I've seen decreased TCP performance on the loopback due to
scheduling interactions with ULE on some systems (but not all -- disabling
checksum generate/verify has improved loopback on other systems).
The first thing to establish is whether other similar benchmarks give the same
result, which might us to narrow the issue down a bit. Could you try using
netperf+netserver with the TCP_STREAM test and see if that differs using the
otherwise identical configuration?
Could you compare the ifconfig link configuration of 7.2 and 8.0 to make sure
there's not a problem with the driver negotiating, for example, half duplex
instead of full duplex? Also confirm that the same blend ot LRO/TSO/checksum
offloading/etc is present.
Could you do "procstat -at | grep ifname" (where ifname is your interface
name) and send that to me?
Another thing to keep an eye of is interrupt rates and pin sharing, which are
both sensitive to driver change and ACPI changes. It wouldn't hurt to compare
vmstat -i rates not just on your network interface, but also on other devices,
to make sure there's not new aliasing. With a new USB stack and plenty of
other changes, additional driver code running when your NIC interrupt fires
would be highly measurable.
Finally, two TCP tweaks to try:
(1) Try disabling in-flight bandwidth estimation by setting
net.inet.tcp.inflight.enable to 0. This often hurts low-latency,
high-bandwidth local ethernet links, and is sensitive to many other issues
including time-keeping. It may not be the "cause", but it's a useful
thing to try.
(2) Try setting net.inet.tcp.read_locking to 0, which disables the read-write
locking strategy on global TCP locks. This setting, when enabled,
significantly impoves TCP scalability when dealing with multiple NICs or
input queues, but is one of the non-trivial functional changes in TCP.
Robert N M Watson
Computer Laboratory
University of Cambridge
More information about the freebsd-current
mailing list