kern/91032: invalid IP checksum under if_bridge(4)+em(4) combination

Ralf S. Engelschall rse at
Wed Dec 28 12:50:05 PST 2005

>Number:         91032
>Category:       kern
>Synopsis:       invalid IP checksum under if_bridge(4)+em(4) combination
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Dec 28 20:50:03 GMT 2005
>Originator:     Ralf S. Engelschall
>Release:        FreeBSD 6.0-STABLE i386
NIC driven by em(4) attached to a bridge based on if_bridge(4).
FreeBSD 6.0-STABLE FreeBSD 6.0-STABLE #0: Wed Dec 28 19:32:06 CET 2005     root at  amd64

I've a HP DL385 running under FreeBSD 6.0-STABLE (as of 2005-12-28)
which has a tap(4) device "tap0" bridged, via the if_bridge(4) device
"bridge0", to the em(4) device "em0".

Without the bridge0 attached to em0, IP packets sent out on em0 have
a correct header checksum. Once the bridge0 is established and em0
attached to it, packets sent out on em0 have an incorrect header
checksum of 0x0000 and this way are just dropped by remote hosts.

The reasons for the 0x0000 checksum is that...

1. if_bridge(4) for unknown reasons explicitly clears the checksum in
   the function if_bridge.c:bridge_enqueue().

2. em(4) for unknown reasons DOES NOT perform the "checksum offloading",
   i.e., calculate the checksum via hardware assistance, if the packets
   comes in via if_bridge(4).

Hence a possible workaround for me was to simply disable the checksum
offloading on "em0" via "ifconfig em0 -txcsum". This effectively solved
the networking problems, but this is just a workaround.

Another workaround would have been to put into the box a 100baseTX NIC
driven by fxp(4) instead of the 1000baseTX NIC driven by em(4). Because
the combination of if_bridge(4) and fxp(4) I've running fine with mostly
the same configuration on another server.

The reason why em(4) doesn't perform the checksum offloading I do not
understand. This might be perhaps a buglet and is perhaps related to
the different packet flow through the system in the cases with and
without if_bridge(4). Perhaps someone who better knows both em(4) and
the internal packet flows can check this.

But the reason why if_bridge(4) _unconditionally_ clears the checksums
of all enqueued packets is totally unclear to me. That a bridge _checks_
the checksums of incoming packets is ok. That a bridge drops packets
with bad checksum I also can accept. But that a bridge clears the
checksum on incoming packets confuses me.

Perhaps it was done because if_bridge(4) not just forwards packets but
also _generates_ new one in case STP is performed. Here if_bridge(4)
perhaps feels lazy and just unconditionally clears the checksum in the
lower level function bridge_enqueue(). But IMHO the correct way would be
to conditionally clear the checksum only for the newly generated packets
(where a new checksum has to be generated) but not for the forwarded
ones (where the checksum already has to exist).

Create a bridge with if_bridge(4) between a em(4) interface and for
instance a tap(4) interface. Then send out packets on em(4) and capture
them. Then look at the IP header and recognize that it contains an
invalid header checksum value of 0x0000.
A workaround is to disable the "checksum offloading" on em(4) with
"ifconfig em0 -txcsum". But the real fix IMHO is to conditionally clear
the checksum in if_bridge(4) only for the newly generated packets and
additionally to figure out why em(4) doesn't perform the checksum
(re-)calculation under "txcsum" if the interface is attached to a
if_bridge(4) device.


More information about the freebsd-bugs mailing list