FreeBSD discarding received packets > MTU

[David Christensen]

> Julian Elischer wrote:
> > Ok which clever person did this again?
> > 
> > It just broke our product.
> > If it hasn't been removed from 7.0 and 6.x yet it's just 
> about to be...
> 
> 
> on the topic above, I've found that the bce driver seems to 
> be throwingaway packets
> larger than the mtu regardless of the rest of the system..

This has been part of the driver since it's initial release,
it's not an addition or change.

> 
> the following code seems to be responsible..
> 
> in the spirit of "be generous on receive" as mentionned before,
>  should it be relaxed?
> 

When this RX MTU size issue was being discussed previously I
was wondering why nobody actually discussed what the drivers 
are doing.  Both the NetXtreme (bge) and NetXtreme II (bce)
controllers have a setting in the RX MAC to specify the MTU
size of the medium.  Anything larger than the MTU is discarded
and a statistic is recorded.  Support for jumbo frames is
available (though not on all bge supported controllers) but the
general understanding from the hardware design perspective is
that the MTU is the maximum message size on the medium and that
all controllers on the medium should be configured for the same
message size.  I've never seen an Ethernet driver that supports
different MTUs for RX and TX (is there one implemented under 
FreeBSD?) and I would really hate to troubleshoot any higher 
level application problems that might occur if the RX and TX MTU 
were different for every system on a network.

> 
> 
>         /* Calculate and program the Ethernet MTU size. */
>         ether_mtu = ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN + 
> ifp->if_mtu +
>                 ETHER_CRC_LEN;
> 
>         DBPRINT(sc, BCE_INFO, "%s(): setting mtu = 
> %d\n",__FUNCTION__, ether_mtu);
> 
> 
>         /*
>          * Program the mtu, enabling jumbo frame
>          * support if necessary.  Also set the mbuf
>          * allocation count for RX frames.
>          */
>         if (ether_mtu > ETHER_MAX_LEN + ETHER_VLAN_ENCAP_LEN) {
>                 REG_WR(sc, BCE_EMAC_RX_MTU_SIZE, ether_mtu |
>                         BCE_EMAC_RX_MTU_SIZE_JUMBO_ENA);
>                 sc->mbuf_alloc_size = MJUM9BYTES;
>         } else {
>                 REG_WR(sc, BCE_EMAC_RX_MTU_SIZE, ether_mtu);
>                 sc->mbuf_alloc_size = MCLBYTES;
>         }
> 
> 
> could probably be relaxed in the non jumbo case to accept 
> packets up to 
> MCLBYTES in size instead of ether_mtu.

Maybe MCLBYTES is good enough for you but what if someone has
a different requirement?  How big is big enough for everyone?

I suppose supporting different MTUs for RX and TX is possible,
though the memory requirements are 5x larger for jumbo frames
over standard frames and many users might consider this a waste
of resources if they don't support jumbo frames on their 
network (that would be almost 4MB).  Support would obviously
be easier if "ifconfig" support both an RX and TX MTU size.

> 
> manually I have set the mtu to larger values which increases
> ether_mtu and that seems to work..
> now, Why do we need this?
> 
> WCCP is a protocol from Cisco.
> it  encapsulates packets in GRE, including an extra 
> intermediate header.
> when packets are encapsulated in GRE by cisco 6k switches 
> (quite a few around)
> they do NOT frag the result, but instead, send an oversized packet.
> 
> if you have a bce interface, it discards these packets which 
> is a drag.

It could certainly be argued by some that Cisco is not standards
compliant in this case for sending an oversized Ethernet frame
and expecting everyone to accept it.  Hardware has limitations
and assuming that all Ethernet controllers can support frames
greater than 1522 bytes is not reasonable.  Fortunately there is
a suitable workaround which is setting a larger MTU for the 
interface.  What size do you use?  How did you arrive at that
value?

Dave