Flow ID, LACP, and igb

[Alexander V. Chernikov]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 26.08.2013 21:18, Justin T. Gibbs wrote:
> Hi Net,
> 
> I'm an infrequent traveler through the networking code and would 
> appreciate some feedback on some proposed solutions to issues
> Spectra has seen with outbound LACP traffic.
> 
> lacp_select_tx_port() uses the flow ID if it is available in the
> outbound mbuf to select the outbound port.  The igb driver uses the
> msix queue of the inbound packet to set a packet's flow ID.  This
> doesn't provide enough
Not sure about igb, but ixgbe (according to advanced RX descriptor
format, 7.1.6.2 @ 82599 datasheet) can provide 'real' RSS value which
can be used in m_flowid instead of NIC queue id.

(And, by the way, another RSS-related problem:
there are cases when setting flowid does more harm, for example -
PPPoE frames always being received at Q0.
Partially this can be solved by analyzing RSS type from the same RX
descriptor format (e.g. don't set flowid for RSS type 0x0), but there
are other cases like GRE tunneling (where you probably want to perform
deeper inspection in SW).

So, can we have some kind of per-NIC sysctl disabling setting flowid
on given port?

(Yes, this should be in some kind of `ethtool` binary but we still
don't have it..)
)

Jack, what do you think?

> bits of information to yield a high quality flow ID.  If, for
> example, the switch controlling inbound packet distribution does a
> poor job, the outbound packet distribution will also be poorly
> distributed.
> 
> The majority of the adapters supported by this driver will compute 
> the Toeplitz RSS hash.  Using this data seems to work quite well in
> our tests (3 member LAGG group).  Is there any reason we shouldn't 
> use the RSS hash for flow ID?
> 
> We also tried disabling the use of flow ID and doing the hash
> directly in the driver.  Unfortunately, the current hash is pretty
> weak.  It multiplies by 33, which yield very poor distributions if
> you need to mod the result by 3 (e.g. LAGG group with 3 members).
> Alan modified the driver to use the FNV hash, which is already in
> the kernel, and this yielded much better results.  He is still
> benchmarking the impact of this change.  Assuming we can get decent
> flow ID data, this should only impact outbound UDP, since the stack
> doesn't provide a flow ID in this case.
> 
> Are there other checksums we should be looking at in addition to
> FNV?
> 
> Thanks, Justin
> 
> _______________________________________________ 
> freebsd-net at freebsd.org mailing list 
> http://lists.freebsd.org/mailman/listinfo/freebsd-net To
> unsubscribe, send any mail to
> "freebsd-net-unsubscribe at freebsd.org"
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (FreeBSD)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlIi9mEACgkQwcJ4iSZ1q2meFQCfQ9QO+y/9ArTXQBAB9RDCGVY2
SpEAnAgZ0vRYuJ0HMamCnpd8q8/yxsLE
=RlDE
-----END PGP SIGNATURE-----