SACK (and PF) wierdness

[Max Laier]

Pawel,

On Saturday 20 November 2004 04:46, Pawel Worach wrote:
> I bumped into a wierd problem with SACK.
>
> Basically my setup is.
> 192.168.1.10    .-crossover             192.168.1.200
> ftp server fxp0<->wireless ap<-> ~~~ <->laptop wireless ath0
>
> I run ftp from the laptop to the server.
> This is what happens:
> ftp> get zero
> local: zero remote: zero
> 200 EPRT command successful.
> 150 Opening BINARY mode data connection for 'zero'.
>     476 KB  299.53 KB/s
> 426 Data connection: Operation not permitted.
> 487424 bytes received in 00:01 (299.49 KB/s)
>
> I started to look at tcpdump while this was happening and quickly
> noticed that the connection got dropped by PF when SACK kicked in.
>
> pf: BAD state: TCP 192.168.1.10:20 192.168.1.10:20 192.168.1.200:50640 
> [lo=3604799807 high=3604800103 win=33304 modulator=0 wscale=1] 
> [lo=4089843176 high=4089909784 win=33304 modulator=0 wscale=1]
> 4:4 FPA seq=3604799807 ack=4089843176 len=296 ackskew=0 pkts=2497:1693 
> dir=out ,fwd    
> pf: State failure on: 1      |

This is an "off by one" due to the FIN flag - I suppose. 3604799807 + 296 is 
3604800103, but the +1 from the FIN flag brings that out of window and causes 
PF to drop the packet.

> Nov 20 04:27:40 <kern.crit> darkstar kernel: pf: BAD state: TCP 
> 192.168.1.10:20 192.168.1.10:20 192.168.1.200:58378 [lo=1373010668
> high=1373010980 win=33304 mod ulator=0 wscale=1] [lo=3742879382
> high=3742945990 win=33304 modulator=0 wscale=1 ] 4:4 A seq=1373010668
> ack=3742879382 len=1448 ackskew=0 pkts=1266:851 dir=out,f wd
> Nov 20 04:27:40 <kern.crit> darkstar kernel: pf: State failure on: 1      
> | Nov 20 04:27:40 <kern.crit> darkstar kernel: pf: BAD state: TCP
> 192.168.1.10:20 192.168.1.10:20 192.168.1.200:58378 [lo=1373010668
> high=1373010980 win=33304 mod ulator=0 wscale=1] [lo=3742879382
> high=3742945990 win=33304 modulator=0 wscale=1 ] 4:4 A seq=1373010668
> ack=3742879382 len=1448 ackskew=0 pkts=1266:851 dir=out,f wd

These two make no sense at all (at least to me). seq + len is over the window 
by 1136 and I don't have the slightest clue why that would be the case. I am 
also a bit surprised that the two (three) state failures are so close 
together (04:27:35 and 04:27:40). Really strange.

> If I disable sack on the ftp server everything works fine.
>
> I can reproduce this problem 100%, I have never managed to transfer more
> than 3Mb via ftp when SACK is on, with it off I see no problems, 11Mbit
> wireless at ~650Kb/s
>
> Attached are three tcpdumps of the ftp data channel after a 'get
> /dev/zero'. (I picked out the smallest ones, dropped after about 400kb of
> zeros)

They didn't make it to the Mailinglist - I am afraid. Can you upload it 
somewhere or try to resend it via private mail? I'd be very interested.

> related pf.conf rules, on ftp server:
> pass out log quick on fxp0 inet proto tcp from fxp0 to any flags S/SA keep
>   state queue (bulk, fast)
> and on client:
> pass in log quick inet proto tcp from any port 20 to <firewall> port >=
> 1024 flags S/SA keep state
>
> Any ideas? More info?

Not yet. But the "off by one" that triggered the first failure should be 
tracked. I am not a TCP-expert myself, so I hope somebody can jump in here. 
Thanks.

-- 
/"\  Best regards,                      | mlaier at freebsd.org
\ /  Max Laier                          | ICQ #67774661
 X   http://pf4freebsd.love2party.net/  | mlaier at EFnet
/ \  ASCII Ribbon Campaign              | Against HTML Mail and News
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-net/attachments/20041121/a4a4b445/attachment.bin