Re: Two different places between TCP socket behavior and RFC documents

On Thu, Sep 18, 2025 at 6:25 PM Michael Tuexen
<michael.tuexen@lurchi.franken.de> wrote:
> > 2. Sending RST to segment with old sequence SYN-RECEIVED instead of
> > acknowledgement
> > According to RFC793 page 69: If an incoming segment is not acceptable, an
> > acknowledgement should be sent in reply. (here `should` is not capitalized).
> > This should be applied to all states including and after SYN-RECEIVED. But it's
> > not the case with FreeBSD TCP socket. I found this with manually constructed TCP
> > segment:
> >  A > B: Flags [S], seq 1, win 8192, length 0
> >  B > A: Flags [S.], seq 4054810353, ack 2, win 65535, length 0
> >  A > B: Flags [.], ack 1, win 8192, length 0
> >  B > A: Flags [R], seq 4054810354, win 0, length 0
> I am not sure which scenario are you considering. Could you provide SEG.SEQ
> for the this TCP segment?
> > Expected behavior is to send an empty ack:
> >  A > B: Flags [S], seq 1, win 8192, length 0
> >  B > A: Flags [S.], seq 3620804602, ack 2, win 65495, length 0
> >  A > B: Flags [.], ack 1, win 8192, length 0
> >  B > A: Flags [.], ack 1, win 65495, length 0
> > Which is the case with Linux.

I'd be happy to explain the scenario in more detail.
Consider the following TCP handshake sequence:
1. Socket A sends a SYN segment: <CTL=SYN><SEQ=x> to Socket B, which is in the
   TCP_LISTEN state.
2. Socket B transitions to TCP_SYN_RECV and responds with
   <CTL=SYN,ACK><SEQ=y><ACK=x+1>.
3. Instead of sending the expected <CTL=ACK><SEQ=x+1><ACK=y+1> to complete the
   three-way handshake, Socket A incorrectly sends <CTL=ACK><SEQ=x><ACK=y+1>.
According to the RFC, the appropriate response to such a malformed ACK should be
an empty ACK segment: <CTL=ACK><SEQ=y+1><ACK=x+1>. After that, Socket B should
either wait for a valid ACK or retransmit the SYN-ACK if necessary.
However, in FreeBSD’s current implementation, a RST segment is sent instead:
<CTL=RST><SEQ=y+1>, which aborts the connection prematurely.
This behavior appears to deviate from the RFC guidance and may lead to
unnecessary connection resets in edge cases.
Best regards
Tilnel