kern/57100: disable hardware checksums when using bridge(4).

[pak at cns.utoronto.ca]

>Number:         57100
>Category:       kern
>Synopsis:       disable hardware checksums when using bridge(4).
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          change-request
>Submitter-Id:   current-users
>Arrival-Date:   Mon Sep 22 10:20:10 PDT 2003
>Closed-Date:
>Last-Modified:
>Originator:     pak
>Release:        FreeBSD 4.8-RELEASE i386
>Organization:
Computing and Network Services, U Toronto
>Environment:
System: FreeBSD host1 4.8-RELEASE FreeBSD 4.8-RELEASE #2: Tue Sep 16 14:18:09 EDT 2003 pak at host1:/usr/src/sys/compile/BOX i386

  - dmesg excerpts:
	   ...
	xl0: <3Com 3c905B-TX Fast Etherlink XL> port [...]
	   ...
	em0: <Intel(R) PRO/1000 Network Connection, Version - 1.7.16> port [...]
	   ...
	BRIDGE 020214 loaded
	   ...

  host1# ifconfig
  xl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
          options=3<rxcsum,txcsum>
          inet 300.2.2.2 netmask 0xffffff00 broadcast 300.2.2.255
		[...]
  em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
          options=3<rxcsum,txcsum>
          inet 10.1.1.1 netmask 0xfffffff8 broadcast 10.1.1.7
		[...]

  host1# sysctl -w net.link.ether.bridge=1

  host1# sysctl -w net.link.ether.bridge_cfg="xl0,em0"

  - xl0 is directly connected to the "outside world."
  - xl0 is configured with a proper IP address (eg. 300.2.2.2 ).

  - em0 is configured with a non-routable RFC-1918 address (say 10.1.1.1).
  - em0 is connected to a switch along with another system called inner1
    so that traffic to and/or from inner1 is being bridged by host1.

  - a rough diagram:
	                 :-----------:
	{  the  }        :   host1   :      |
	{outside} ----- xl0         em0 ----+
	{ world }        :           :      |
	                 :-----------:      |    :--------:
	                                    +--- : inner1 :
	                                    |    :--------:

>Description:

  When bridging is enabled, hardware checksumming by the NICs corrupts
  the checksums for packets which are generated by the bridging system
  and intended for systems "on the inside".  In the example, with
  bridging enabled, inner1 is able to talk to the outside world but it's
  not able to talk to host1 via host1's "proper" address. IP packets 
  generated by host1 and intended for inner1 have incorrect checksums.
  Disabling hardware checksums during bridging appears to fix this.


>How-To-Repeat:
>Fix:

  Included below is a patch to disable (restore) the use of hardware
  checksums on bridging NICs when bridging is enabled (disabled).
  The same sort of manipulation of 'hwassist' can be found
  in /sys/netgraph/ng_ether.c.  Hope this helps.  pak.


*** net/bridge.c	2003/09/22 15:09:35	1.1
--- net/bridge.c	2003/09/22 14:51:23
***************
*** 314,319 ****
--- 314,320 ----
  
  	if ( b->flags & IFF_BDG_PROMISC ) {
  	    s = splimp();
+ 	    ifp->if_hwassist = b->hwassist;	/* restore hwassist. */
  	    ifpromisc(ifp, 0);
  	    splx(s);
  	    b->flags &= ~(IFF_BDG_PROMISC|IFF_MUTE) ;
***************
*** 361,366 ****
--- 362,369 ----
  	if ( !(b->flags & IFF_BDG_PROMISC) ) {
  	    int ret ;
  	    s = splimp();
+ 	    b->hwassist = ifp->if_hwassist; 	/* save hwassist. */
+ 	    ifp->if_hwassist = 0;		/* disable hwassist. */
  	    ret = ifpromisc(ifp, 1);
  	    splx(s);
  	    b->flags |= IFF_BDG_PROMISC ;
*** net/bridge.h	2003/09/22 14:52:05	1.1
--- net/bridge.h	2003/09/22 14:53:12
***************
*** 47,52 ****
--- 47,53 ----
  #define IFF_BDG_PROMISC 0x0001  /* set promisc mode on this if.	*/
  #define IFF_MUTE        0x0002  /* mute this if for bridging.   */
  #define IFF_USED        0x0004  /* use this if for bridging.    */
+     u_long hwassist;		/* hardware checksum capabilities */
      struct cluster_softc *cluster;
  } ;
  


>Release-Note:
>Audit-Trail:
>Unformatted: