svn commit: r327559 - in head: . sys/net

[Steven Hartland]

On 05/01/2018 02:09, Eugene Grosbein wrote:
> 05.01.2018 6:37, Steven Hartland wrote:
>
>> Our TCP stack seems fragile during setup to out of order packets
>> which this multipath behavior causes, we've seen this on our loadbalancers
>> which is what triggered the investigation. The concrete result is many aborted TCP connections,
>> over 300k ~2% on the machine I'm looking at.
> This is another problem that needs to be fixed in general and not hidden under the carpet.
> Meantime, practical problems you see can be solved locally with any settings you like.
While it may seem like it, there's not denying that the problem is 
caused by fact that the packets for a single flow arrive on two 
different interfaces in normal (none failure) workflow, which 
contravenes 802.3ad which states:

43.2.4 Frame Distributor
…
This standard does not mandate any particular distribution algorithm(s); 
however, any distribution algorithm shall ensure that, when frames are 
received by a Frame Collector as specified in 43.2.3, the algorithm 
shall not cause
a) Mis-ordering of frames that are part of any given conversation, or
b) Duplication of frames.
The above requirement to maintain frame ordering is met by *ensuring 
that all frames that compose a given conversation are transmitted on a 
single link in the order* that they are generated by the MAC Client; 
hence, this requirement does not involve the addition (or modification) 
of any information to the MAC frame, nor any buffering or processing on 
the part of the corresponding Frame Collector in order to re-order frames.

>
>> I hope there's some improvements that can be made, for example if we can determine
>> the stream was instigated remotely then flowid would always be valid hence we can use it assuming it
>> matches the requested spec or if we can make it clear to the user that laggproto is not the one they requested, I'm open to ideas?
> We just need to clear flow id from incoming TCP segments and always generate new flow id for responses
> keeping old flow id for IP forwarding case. Please back out your change to not degrade IP forwarding performance.
Sorry I don't follow you. You seem to be inferring that we can easily 
generate a flowid without involving the sending hardware; I can't see 
how that is possible as that's chicken and egg i.e. you can't get the HW 
interface to generate the flowid without sending a packet and you can't 
send a packet until you have a the flowid to decide which interface to 
send it from.

     Regards
     Steve