FreeBSD discarding received packets > MTU

[Julian Elischer]

David Christensen wrote:
>> Julian Elischer wrote:
>>> Ok which clever person did this again?
>>>
>>> It just broke our product.
>>> If it hasn't been removed from 7.0 and 6.x yet it's just 
>> about to be...
>>
>>
>> on the topic above, I've found that the bce driver seems to 
>> be throwingaway packets
>> larger than the mtu regardless of the rest of the system..
> 
> This has been part of the driver since it's initial release,
> it's not an addition or change.
> 
>> the following code seems to be responsible..
>>
>> in the spirit of "be generous on receive" as mentionned before,
>>  should it be relaxed?
>>
> 
> When this RX MTU size issue was being discussed previously I
> was wondering why nobody actually discussed what the drivers 
> are doing.  Both the NetXtreme (bge) and NetXtreme II (bce)
> controllers have a setting in the RX MAC to specify the MTU
> size of the medium.  Anything larger than the MTU is discarded
> and a statistic is recorded.  Support for jumbo frames is
> available (though not on all bge supported controllers) but the
> general understanding from the hardware design perspective is
> that the MTU is the maximum message size on the medium and that
> all controllers on the medium should be configured for the same
> message size.  I've never seen an Ethernet driver that supports
> different MTUs for RX and TX (is there one implemented under 
> FreeBSD?) and I would really hate to troubleshoot any higher 
> level application problems that might occur if the RX and TX MTU 
> were different for every system on a network.
> 
>>
>>         /* Calculate and program the Ethernet MTU size. */
>>         ether_mtu = ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN + 
>> ifp->if_mtu +
>>                 ETHER_CRC_LEN;
>>
>>         DBPRINT(sc, BCE_INFO, "%s(): setting mtu = 
>> %d\n",__FUNCTION__, ether_mtu);
>>
>>
>>         /*
>>          * Program the mtu, enabling jumbo frame
>>          * support if necessary.  Also set the mbuf
>>          * allocation count for RX frames.
>>          */
>>         if (ether_mtu > ETHER_MAX_LEN + ETHER_VLAN_ENCAP_LEN) {
>>                 REG_WR(sc, BCE_EMAC_RX_MTU_SIZE, ether_mtu |
>>                         BCE_EMAC_RX_MTU_SIZE_JUMBO_ENA);
>>                 sc->mbuf_alloc_size = MJUM9BYTES;
>>         } else {
>>                 REG_WR(sc, BCE_EMAC_RX_MTU_SIZE, ether_mtu);
>>                 sc->mbuf_alloc_size = MCLBYTES;
>>         }
>>
>>
>> could probably be relaxed in the non jumbo case to accept 
>> packets up to 
>> MCLBYTES in size instead of ether_mtu.
> 
> Maybe MCLBYTES is good enough for you but what if someone has
> a different requirement?  How big is big enough for everyone?
> 
> I suppose supporting different MTUs for RX and TX is possible,
> though the memory requirements are 5x larger for jumbo frames
> over standard frames and many users might consider this a waste
> of resources if they don't support jumbo frames on their 
> network (that would be almost 4MB).  Support would obviously
> be easier if "ifconfig" support both an RX and TX MTU size.
> 
>> manually I have set the mtu to larger values which increases
>> ether_mtu and that seems to work..
>> now, Why do we need this?
>>
>> WCCP is a protocol from Cisco.
>> it  encapsulates packets in GRE, including an extra 
>> intermediate header.
>> when packets are encapsulated in GRE by cisco 6k switches 
>> (quite a few around)
>> they do NOT frag the result, but instead, send an oversized packet.
>>
>> if you have a bce interface, it discards these packets which 
>> is a drag.
> 
> It could certainly be argued by some that Cisco is not standards
> compliant in this case for sending an oversized Ethernet frame
> and expecting everyone to accept it.  Hardware has limitations
> and assuming that all Ethernet controllers can support frames
> greater than 1522 bytes is not reasonable.  Fortunately there is
> a suitable workaround which is setting a larger MTU for the 
> interface.  What size do you use?  How did you arrive at that
> value?

I use 1550 to make it work in the test harness.

The trouble is that if I set the mtu to 1550, and the machine talks to another
such machine with it's mtu also set to 1550 then they negotiate a maximum sized
packet based on 1550, and the problem hits me again. This is a web proxy 
and that problem occurs when there are two layers of proxy and one proxy talks to 
another. I really just need it to to silently accept a packet some 
32 bytes or so larger than the stated MTU.

I see no reason for the driver to not do what the em driver does and allow 
itself to receive any packet up to the MCLBYTES size.

We only hit this problem recently because the data interfaces on our
devices are usually em NICs and we only just recently started allowing the 
users to use the built in (on DELL 2950) bce interfaces for this purpose.


> 
> Dave