FreeBSD 5.4+SMP, severe network degredation
dpk at dpk.net
Thu Jul 28 03:26:09 GMT 2005
We just received several SuperMicro servers, 3.0Ghz Xeon x 2, 4GB RAM.
They're using the em driver and the ports are set to 1000Mbit (we also
tried 100Mbit/full duplex on the card and on the switch). They're running
I ran a steady ping on a couple of them while they were running "GENERIC",
and then rebooted them with a kernel built with the "PAE" kernel included
with the installation, with "option SMP" added.
The PAE-SMP-GENERIC kernel was built after cvsup'ing with "tag=RELENG_5_4"
and the uname reports "5.4-RELEASE-p5".
Here are the ping results:
117 packets transmitted, 117 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.451/0.554/0.856/0.059 ms
102 packets transmitted, 102 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.569/4.262/7.944/2.065 ms
Fetching a 637MB ISO from a local server, also on 100/FDX:
/dev/null 100% of 637 MB 10 MBps 00m00s
/dev/null 100% of 637 MB 5764 kBps 00m00s
Running GENERIC, systat shows about 7000 interrupts/second, and around 600
interrupts/second using PAE-SMP-GENERIC, while fetch was running.
I've checked the errata and hardware notes, as well as gnats, and was not
able to find anything that explains or matches this behavior. We've run
SMP servers for years, using 4.5-4.11, but we've never seen the network
performance cut in half (or pings go up 10x).
Removing "option SMP" makes the problem go away, but at a very significant
performance cost obviously.
Could it be something from -p5? Is this explained/examined in a PR I've
missed, and if so can I add some information?
More information about the freebsd-questions