Problem with mxge on RELENG_8_1

[Garance A Drosihn]

At 3:21 PM -0400 8/13/10, Robert Healey wrote:
>
>I recently updated the central file server/router systems for a
>pair of research clusters from RELENG_8_0 to RELENG_8_1.  After
>following the proper procedures, the network throughput when
>pulling files from both machines via mxge0 is 200KB/s or less.
>Before the update, 50MB/s was the normal rate.
>
>Doing some testing, with the 8.1 kernel booted, I can upload
>files to the server over the 10G link with scp or NFS at the
>expected rates.  I can fetch files from the Internet using
>this server as the NAT gateway also at the expected rates.
>Retrieving files from the server over mxge, the throughput
>falls to 200KB/s.  Retrieving files from the server from the
>onboard Broadcom NIC, rates are as to be expected from gigabit.
>With 8.0, everything works as expected.
>
>Hardware Config 1:
>Dell Poweredge R610 with 2 Xeon E5530 @ 2.4 GHz, hyperthread
>disabled and 24GB RAM.  Onboard interface is bce. Disk is
>attached via a PERC 6/E. Internal cluster switch is Dell
>Powerconnect 6248P.  This switch sees excessive large packets
>on the 10G connection on 8.1, but not 8.0.
>
>Hardware Config 2:
>HP Proliant DL 320G6 with 1 Xeon  E5540 @ 2.53GHz, hyperthread
>enabled and 8GB RAM.  Onboard interface is bge.  Disk is
>attached via a HP Smart Array P212.  Internal cluster switch
>is an HP Procurve 2910al.  It does not see any packet errors
>from the 10G link.


You're seeing the performance problems with both scp and NFS?

Could you get some tcpdumps of a file transfer with both setups
(8.0 vs 8.1), just to see if something very odd jumps out?

If I'm reading this right, file transfers work at expected speeds
if the file is going *to* the server, but you see the slowdown
when you're pulling files *from* the server.  That sounds similar
to a problem with a duplex-mismatch, except of course that you
shouldn't be getting into duplex issues on a gigabit network.

-- 
Garance Alistair Drosehn            =   gad at gilead.netel.rpi.edu
Senior Systems Programmer           or  gad at freebsd.org
Rensselaer Polytechnic Institute    or  drosih at rpi.edu