TCP out-of-order packets.

[Julian Elischer]

Andre Oppermann wrote:

>Julian Elischer wrote:
>  
>
>>Brooks Davis wrote:
>>
>>    
>>
>>>On Wed, Jan 12, 2005 at 05:07:36PM -0800, Julian Elischer wrote:
>>>
>>>
>>>      
>>>
>>>>I have a link which is provided by someone else that is 7 x E1s aggregated.
>>>>At leat it looks that way to me when I get to see it. however I have
>>>>only been able to get
>>>>60kB.sec across this, despite having a tcp window size of 131072 bytes..
>>>>After investigation it appears that the link is massively re-orderring
>>>>packets.
>>>>groups of upto 10 packets may appear in random order. (Maybe more, bu tI
>>>>have seen 10)
>>>>
>>>>in fact packets are rarely IN order.
>>>>
>>>>This plays havoc with the tcp sessions.
>>>>
>>>>I was thinking of writing a hacked up version of NATD that
>>>>instead of doing NAT, just did a pre-sort on packets from each session,
>>>>so that the receiver would
>>>>see a stream of IN-order packets, with occasional delays.
>>>>
>>>>firstly, does anyone have any tools to do this already (why build when
>>>>you can borrow)
>>>>and secondly, does anyone have any experience with this sort of problem?
>>>>
>>>>I have no control over or access to the link.. all I have is a promise
>>>>that they will deliver
>>>>14Mb/Sec. with approc 300mSec. RTT to me but there is no promise about
>>>>packet order.
>>>>
>>>>I just get a 100Mb ethernet cable.
>>>>
>>>>
>>>>        
>>>>

>>>>
>>>>I wonder if there's a way to turn off the sender backoff?
>>>>        
>>>>
>
>Not directly.  What you actually want is to delay the generating of
>ACK's for a certain amount of time (some milli-seconds) to aggregate
>the out-of-order packets into one ACK and to avoid the backoff from
>the other side.
>  
>

yes, here's a trace from the receiver's end showing the order of 
received packets..
RTT on this link is about 300mSec.

My original answer would be to hack NATD to re-order the packets.
and use DIVERT to it.

DeltaT  SEQ-SRTR:SEQ_END    packet#
------------------------------------------
346853 2856:2920(64)            1
370821 2920:4368(1448)          2
004410 8712:10160(1448)         6
007848 12608:14056(1448)        9
007057 18400:19848(1448)        13
004027 11160:12608(1448)        8
005553 4368:5816(1448)          3
002390 16952:18400(1448)        12
005745 7264:8712(1448)          5
001204 5816:7264(1448)          4
008326 14056:15504(1448)        10
002555 10160:11160(1000)        7
004091 19848:21296(1448)        14
004287 15504:16952(1448)        11
358289 22744:24192(1448)        16
001891 21296:22744(1448)        15
048289 4368:5816(1448)          DUP of 3
025538 5816:7264(1448)          DUP of 4
018761 10160:11608(1448)        DUP of 7
062939 25640:27088(1448)        18
005493 15504:16952(1448)        DUP of 11
007903 24192:25640(1448)        17
002207 27088:28536(1448)        19
028675 21296:22744(1448)        DUP of 15
176400 28536:29984(1448)        20
020889 29984:31432(1448)        21
092304 24192:25640(1448)        DUP of 17
038396 31432:32880(1448)        22
015146 32880:34328(1448)        23
010362 27088:28536(1448)        DUP of 19
027400 35776:37224(1448)        25
008269 28536:29984(1448)        DUP of 20
012537 34328:35776(1448)        24
294035 37224:38672(1448)        26
078059 34328:35776(1448)        DUP of 24
267701 40120:41568(1448)        28
017393 38672:40120(1448)        27
019769 41568:43016(1448)        29
339347 44464:45912(1448)        31
003887 43016:44464(1448)        30
123755 45912:47360(1448)        32
228883 47360:48808(1448)        33
013437 48808:50256(1448)        34
219579 50256:51704(1448)        35
144501 51704:53152(1448)        36
001200 53152:54600(1448)        37
029175 54600:56048(1448)        38
183158 56048:57496(1448)        39
157127 58944:60392(1448)        40

By here the sender has backed right off.


>
>  
>