tap on lagg ?

Tue Mar 21 20:15:33 UTC 2017

Hi,

So all in all, I added « -vlanhwtag -txcsum -txcsum6 » on all 4 igb interfaces. They still all put themselves in promiscuous mode, however. But TCPing the host from a VM now works. Please find the relevant dmesg and ifconfig output below.

But I’d still appreciate more background information on this. is « -vlanhwtag » still required ? Should I expect a traffic slowdown on these interfaces ? Will it bring the CPU to its knees ? Why do I still get « bridge0: error setting interface capabilities on lagg0 » ?

Thanks!

Vincent

lagg0: link state changed to UP
igb3: link state changed to UP
igb1: link state changed to UP
igb2: link state changed to UP
bridge0: Ethernet address: 02:f7:d6:01:1a:00
bridge0: link state changed to UP
igb0: promiscuous mode enabled
igb1: promiscuous mode enabled
igb2: promiscuous mode enabled
igb3: promiscuous mode enabled
lagg0: promiscuous mode enabled
tap0: Ethernet address: 00:bd:29:bf:f8:00
bridge0: error setting interface capabilities on lagg0
tap0: promiscuous mode enabled
tap0: link state changed to UP
tap1: Ethernet address: 00:bd:1a:e6:f8:01
bridge0: error setting interface capabilities on lagg0
tap1: promiscuous mode enabled
tap1: link state changed to UP

igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=2403a9<RXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6>
	ether 54:a0:50:88:88:c6
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet 1000baseT <full-duplex>
	status: active
igb1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=2403a9<RXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6>
	ether 54:a0:50:88:88:c6
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet 1000baseT <full-duplex>
	status: active
igb2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=2403a9<RXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6>
	ether 54:a0:50:88:88:c6
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet 1000baseT <full-duplex>
	status: active
igb3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=2403a9<RXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6>
	ether 54:a0:50:88:88:c6
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet 1000baseT <full-duplex>
	status: active
cxl0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	ether 00:07:43:37:47:70
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet none
	status: no carrier
cxl1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
	options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	ether 00:07:43:37:47:78
	inet 192.168.11.5 netmask 0xffffff00 broadcast 192.168.11.255 
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet 10Gbase-Twinax <full-duplex>
	status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128 
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7 
	inet 127.0.0.1 netmask 0xff000000 
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	groups: lo 
lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=2403a9<RXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6>
	ether 54:a0:50:88:88:c6
	inet 192.168.1.23 netmask 0xffffff00 broadcast 192.168.1.255 
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect
	status: active
	groups: lagg 
	laggproto lacp lagghash l2,l3,l4
	laggport: igb0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
	laggport: igb1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
	laggport: igb2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
	laggport: igb3 flags=0<>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: vm-lan1g
	ether 02:f7:d6:01:1a:00
	nd6 options=1<PERFORMNUD>
	groups: bridge 
	id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
	maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
	member: tap1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 11 priority 128 path cost 2000000
	member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 10 priority 128 path cost 2000000
	member: lagg0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 8 priority 128 path cost 6666
tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: vmnet-docker-0-lan1g
	options=80000<LINKSTATE>
	ether 00:bd:29:bf:f8:00
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect
	status: active
	groups: tap 
	Opened by PID 1417
tap1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: vmnet-unifi-0-lan1g
	options=80000<LINKSTATE>
	ether 00:bd:1a:e6:f8:01
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet autoselect
	status: active
	groups: tap 
	Opened by PID 1698

> Le 21 mars 2017 à 12:10, Harry Schmalzbauer <freebsd at omnilan.de> a écrit :
> 
> Bezüglich Vincent Olivier's Nachricht vom 21.03.2017 16:51 (localtime):
>> Hi, I can confirm that ping works but ssh and (I haven’t tried anything else, but I assume you are right) TCP as a whole doesn’t work.
>> 
>> From there, I guess that, since I haven’t changed the MTU on the 1G interfaces (only on the 10G ones which are isolated from the 1G network). this leaves offloading.
>> 
>> Should I disable it (which ones)? On all the physical interfaces or also on the lagg and maybe bridge?
> 
> You seem to have the following problem:
> if_bridge(4) tries to disable TXCSUM on all members added.
> But you add if_lagg(4), which doesn't pass those requests to it's
> members, but simply ignores the request.
> So you need to manually -txcsum (-txcsum6), e.g. in rc.conf when you set
> them "up".
> 
> Unofrtunately I don't know how offloading is implemented generally, nor
> how it works for if_igb(4), so I haven't thought about the reason yet,
> why you need to disable TXCSUM.
> Much more important, does it also affect TSO?
> I can't tell, maybe someone with more knowledge can jump in.
> 
> -harry
> 