ipfw kernel NAT performance much worse in 11-Stable than 10-Stable

[Ian Smith]

On Thu, 31 Aug 2017 15:27:47 +0300, Andrey V. Elsukov wrote:
 > On 31.08.2017 15:10, Graham Menhennitt wrote:
 > > On 10-Stable, the interface is re1. The output of 'ifconfig re1 | grep
 > > options' is:
 > > options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
 > > 
 > >     nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
 > > 
 > > On 11-Stable (the one with the problems), it's igb1 and the output of
 > > 'ifconfig igb1 | grep options' is:
 > > options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
 > > 
 > >     nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
 > > 
 > 
 > You need to disable TSO on your interface, ipfw nat is not compatible
 > with TCP segmentation offloading (this is noted in ipfw(8) BUGS section).
 > 
 > Try to use:
 > ifconfig igb1 -vlanhwtso -tso4
 > 
 > You can add these option to "ifconfig_igb1" variable in rc.conf.

Specifically:

     Due to the architecture of libalias(3), ipfw nat is not compatible with
     the TCP segmentation offloading (TSO).  Thus, to reliably nat your net-
     work traffic, please disable TSO on your NICs using ifconfig(8).

Since natd also uses libalias, does not that also apply when using natd?  
I forget, and neither libalias(3) nor natd(8) mentions 'tso|TSO'.

Since this comes up so often, including on questions@, I'm wondering if 
an extra test in /etc/rc.d/ipfw at ipfw_prestart() for enablement of 
either $natd_enable (if applicable) or $firewall_nat_enable could then 
and there check ifconfig $natd_interface and/or $firewall_nat_interface 
for the presence of TSO4 and/or VLAN_HWTSO options, and so could warn 
the user - or just run "ifconfig $iface -vlanhwtso -tso4" directly?

While some interfaces such as ngX or pppX need not be up or even exist 
when starting ipfw, such interfaces should never use TSO anyway?  But 
I'm probably missing something obvious ..

cheers, Ian