if_em, legacy nic and GbE saturation

[Adrian Chadd]

Hi,

There's bus limits on how much data you can push over a PCI bus. You can
look around online to see what 32/64 bit, 33/66MHz PCI throughput estimates
are.

It changes massively if you use small versus large frames as well.

The last time I tried it i couldn't hit gige on PCI; I only managed to get
to around 350mbit doing TCP tests.


-adrian



On 26 August 2013 00:06, Harald Schmalzbauer <h.schmalzbauer at omnilan.de>wrote:

>  Hello,
>
> I recycled an older box and put an i350-2 together with a second 82541GI
> (PCI-slot, one already on-board) into it.
> The two i350-ports are used with VMDq for ESXi5.1.
> The two 82541GI are used as lagg-nics by a 9.2-RC (amd64) guest as
> passthrou PCI device.
> Always had good results with such setups, but found out, that nics which
> use the legacy driver part of if_em max out at ~0.6Gbits/s (1500 MTU).
>
> There's another NIC on board of this recycle-box, a 82566-PHY (ICH9
> integrated MAC).
> This one uses also if_em, but not legacy code, it reports version 7.3.8
> (compared to 1.0.6).
> And it has no problem fully saturating GbE (~925Mbits/s, no jumbo Frames
> support anyways).
>
> I'm using iperf, with and without lagg (doesn't change anyhing, like it
> doesn't influence tests on some other boxes with 82576 and i350 (igb))
> I see enough idle cycles so CPU shouldn't limit the legacy if_em nics.
> Also, I see the 82541 consuming arround 8k irqs. Same does the
> 82566-PHY, but with much higher throughput...
>
> I'd like to know if I can't generally expect to saturate older (PCI) GbE
> nics the line for any reason... I can remember tigeon cards from more
> than a decade ago, which indeed seemd to lack the performance to gain
> GbE, but I thought that was no issue shortly later and no "modern"
> Intel-GbE card had such constraints!?
> Is there any special tuning for legacy if_em (no need for any TCP
> tuning, 82566 doesn't have any issue)?
>
> Thanks,
>
> -Harry
>
>