[SCTP] ICMP unreachable message reenables data transmit

[Schoch Christian]

Zitat von Michael Tüxen:

> On May 1, 2011, at 1:10 PM, Schoch Christian wrote:
>
>>> On Apr 30, 2011, at 12:15 PM, Schoch Christian wrote:
>>>>
>>>>> On Apr 30, 2011, at 9:11 AM, Schoch Christian wrote:
>>>>>
>>>>>> During a measurement with CMT-SCTP and PF i figured out, that  
>>>>>> sometimes a ICMP Destination unreachable message triggers a  
>>>>>> message transmission on an inactive data path that has been  
>>>>>> primary before.
>>>>>>
>>>>>> It looks as the ICMP message is reseting the inactive state  
>>>>>> back to active without reseting RTO.
>>>>>>
>>>>>> This behavior is triggered by a returning heartbeat message  
>>>>>> when no ICMP unreachable by data is sent quite before.
>>>>>>
>>>>>> Test system are two multi-homed hosts with FreeBSD8.1 and a  
>>>>>> WANem host between.
>>>>>>
>>>>>> A wireshark log can be provided on demand (quite large).
>>>>> Hi Christian,
>>>>>
>>>>> any chance to upgrade the FreeBSD machines to head or to use newer
>>>>> SCTP sources, which I could provide? It would require a recompilation
>>>>> of the kernel...
>>>>
>>>> It is possible, but the results could be provided not until next week
>>>> if a reboot is necessary.
>>>> I can use any sources you could provide me since nothing else is  
>>>> done at this systems.
>>> OK, but maybe I can try to understand what is going on.
>>>
>>> How many paths do you have? One is inactive, but was primary, so it
>>> is confirmed. On another one, you get an ICMP (which one? Port unreachable,
>>> host unreachable, ...). Do you have more than two paths?
>>
>> Setup looks like this:
>>
>>      --------     ----cut---
>> Host A        WANem          Host B
>>      --------     ----------
>>
>> Transfer is running on both path from A to B till the primary link  
>> is cut between WANem and the receiver and the whole transfer  
>> switches to the second path. The ICMP message (Host not reachable  
>> with a Heartbeat as attachment) is received on the primary  
>> interface from WANem host.
> OK, understood.
>>
>> As I tested this morning, the primary path is switching to  
>> unreachable due to the ICMP message but should be in this state  
>> quite before by exceeding path.max_retrans.
>> So this ICMP message does two things:
>> - Set the primary path to unreachable
>> - Triggers something to retry data transfer on the primary path.
> After looking at the tracefile, I somewhat agree.
> * Do you see something like
>   ICMP (thresh ??/??) takes interface ?? down
>   on the console? This would be printed if the ICMP takes the
>   path to unreachable? (It should also be in /var/log/messages)
> * If the path is already unreachable, nothing should happen
>   in response to the ICMP message.
>
  08:04:18  kernel: ICMP (thresh 2/3) takes interface 0xc4e20510 down
Same timestamp as the faulty start in the tracefile.

> So the question is: Is the path unreachable before the ICMP message
> is received?
Due to the timely difference between first retransmission and ICMP  
message it should be in unavailable state. But it seams that too many  
retransmission occur and the ICMP message is moving the path to  
unavailable state.

I picked my eyes to the RTO of primary path and could figure out the  
following:

inital state: rto.min = 100ms
RTO = 100ms

after cutting the link:
RTO rises to 200ms and 400ms as expected but not higher (rto.max=60000)

Another test with path_rxt_max = 1 worked as expected.

So I assume some problems with the retransmission counter when larger  
than 1 (something like count = 1 instead of count >= 1)

> Is your application monitoring the SCTP notification?
> What about the above printout from the kernel?

Yes, the notifications are monitored and logged (sctp_menu) - the  
notification for SCTP_PEER_ADDR_CHANGE comes right after ICMP.

Best regards,
Christian

> Best regards
> Michael
>>
>>> The ICMP message would not reset the RTO, since you need an ACKed TSN
>>> or a HB-ACK to to that. Since it is inactive, it is missing these.
>>>
>>> Sending on an inactive path is OK, as soon as you enter the dormant
>>> state, which means all your paths are inactive.
>>>
>> Transfer is still running on second link which is active.
> That sounds good.
>>
>>> Are you using the PF support for CMT?
>>
>> Yes, but without NR-SACK and DAC.
> OK.
>>
>> I uploaded the pcap file to:
>> http://37116.vs.webtropia.com/cmt_2.pcap
> That was helpful!
>>
>> Best regards,
>> Christian
>>
>>>
>>> Best regards
>>> Michael
>>>>
>>>>>
>>>>> Are you using IPv4 or IPv6?
>>>>>
>>>>
>>>> IPv4
>>>>
>>>>
>>>>> Best regards
>>>>> Michael
>>>>>>
>>>>>> Regards,
>>>>>> Schoch Christian
>>>>>> _______________________________________________
>>>>>> freebsd-net at freebsd.org mailing list
>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> freebsd-net at freebsd.org mailing list
>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>>
>
>