ftp problem

Fri Jan 8 22:51:01 UTC 2010

On Fri, Jan 8, 2010 at 9:50 PM, M. Keith Thompson
<m.keith.thompson at gmail.com> wrote:
> It looks like it was a tcp windowing problem.  The command: "sysctl -w
> sysctl net.inet.tcp.rfc1323=0"
> fixed the problem.

This only fixes a symptom. :) There is something wrong with your ruleset.

>>> # Allow ftp
>>> pass in quick on $ext_if proto tcp from any to $ext_IP port 21 keep state
>>> pass in quick on $ext_if proto tcp from any to $ext_IP port > 49151 keep state
>>> pass in quick on $ext_if proto tcp from any port > 10000 to $ext_IP
>>> port 20 keep state

With this active ftp rule you had before, the HP client could initiate
and transfer over an active ftp data channel,
while this rule does not allow this at all.
There shouldn't be any active ftp datachannel communication possible
because port 20 is the source port and not the destination port.

I tried the following pf.conf , with a similar ftp-data rule , on a
local FreeBSD 7.2 box in my network

EXT_IF=fxp0

# net.inet.ip.portrange.hilast: 65535
# net.inet.ip.portrange.hifirst: 49152
PASSIVE='49152:65535'

set skip on lo0

# default policy
block log all

# outgoing DNS requests

pass out quick on $EXT_IF inet proto udp from $EXT_IF:network to any
port domain keep state
pass out quick on $EXT_IF inet proto tcp from $EXT_IF:network to any
port domain keep state flags S/SA

# incoming SSH

pass in quick on $EXT_IF inet proto tcp from $EXT_IF:network to port
2022 keep state flags S/SA

# incoming FTP

# ftp command channel
pass in quick on $EXT_IF inet proto tcp from any to $EXT_IF port ftp
keep state flags S/SA

# ftp data channel (passive)
pass in quick on $EXT_IF inet proto tcp from any to $EXT_IF port
$PASSIVE keep state flags S/SA

# ftp data channel (active)
# wrong direction!!!
pass in quick on $EXT_IF inet proto tcp from any to $EXT_IF port
ftp-data keep state flags S/SA

Here I also erroneously used port 20 as destination port.

An active ftp command to upload a file aborts with

200 EPRT command successful.
425 Can't build data connection: Operation not permitted.

A pflog0 tcpdump shows the reason:

tcpdump -eni pflog0 -s0 -v
tcpdump: WARNING: pflog0: no IPv4 address assigned
tcpdump: listening on pflog0, link-type PFLOG (OpenBSD pflog file),
capture size 65535 bytes
22:16:42.551221 rule 0/0(match): block out on fxp0: (tos 0x8, ttl 64,
id 2766, offset 0, flags [DF], proto TCP (6), length 60)
192.168.222.244.20 > 192.168.222.20.62316: S, cksum 0xc222 (correct),
488073960:488073960(0) win 65535 <mss 1460,nop,wscale
3,sackOK,timestamp 6278055 0>

So this rule set  blocks the first packet of the three-way TCP handshake.

  192.168.222.244.20 > 192.168.222.20.62316.
               or
   server.20 > client.62316

Your previous ruleset somehow created state or allowed the server to
initiate the data channel to the client somewhere else in your rule
set. It didn't create state on the initial packet of the TCP
handshake, where the TCP window size is negotiated. Just like the
Daniel Hartmeier article states, this a caused tcp window scaling
issue.

Remember that in pf the last matching rule wins.
You only can defeat this behaviour by using  the 'quick' keyword.

After adding the following corrected rule the transfer proceeds
without any blockage

# correct direction/source port
pass out quick on $EXT_IF inet proto tcp from any  port ftp-data to
any keep state flags S/SA

The rules as parsed are now:

# pfctl -vvf /etc/pf.conf
No ALTQ support in kernel
ALTQ related functions disabled
Loaded 696 passive OS fingerprints
EXT_IF = "fxp0"
PASSIVE = "49152:65535"
set skip on { lo0 }
@0 block drop log all
@1 pass out quick on fxp0 inet proto udp from 192.168.222.0/24 to any
port = domain keep state
@2 pass out quick on fxp0 inet proto tcp from 192.168.222.0/24 to any
port = domain flags S/SA keep state
@3 pass in quick on fxp0 inet proto tcp from 192.168.222.0/24 to any
port = down flags S/SA keep state
@4 pass in quick on fxp0 inet proto tcp from any to 192.168.222.244
port = ftp flags S/SA keep state
@5 pass in quick on fxp0 inet proto tcp from any to 192.168.222.244
port 49152:65535 flags S/SA keep state
@6 pass in quick on fxp0 inet proto tcp from any to 192.168.222.244
port = ftp-data flags S/SA keep state
@7 pass out quick on fxp0 inet proto tcp from any port = ftp-data to
any flags S/SA keep state

Rule 6 is the rule with the wrong direction/port specification.
Rule 7 is the corrected one

pflog0 doesn't show any errors:
tcpdump: listening on pflog0, link-type PFLOG (OpenBSD pflog file),
capture size 65535 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel

And a quick state check still showed some remnants of the connections

# pfctl -ss
No ALTQ support in kernel
ALTQ related functions disabled
[snip]
all tcp 192.168.222.244:21 <- 192.168.222.20:13497       FIN_WAIT_2:FIN_WAIT_2
all tcp 192.168.222.244:20 -> 192.168.222.20:55246       FIN_WAIT_2:FIN_WAIT_2
all tcp 192.168.222.244:20 -> 192.168.222.20:59354       FIN_WAIT_2:FIN_WAIT_2

The first state is the ftp command channel client:13497 > server:21
The last two are the active data channels: server:20 > client:55246,
and server.20 > client.59354

Don't get confused, the last octet of the client IP (192.168.222.20)
happens to be the same as
the ftp-data port number. ;)

A repeat of the transfer shows the following verbose state

#  pfctl -vvss

all tcp 192.168.222.244:20 -> 192.168.222.20:64857       ESTABLISHED:ESTABLISHED
   [1427346948 + 65536] wscale 3  [200059941 + 37224] wscale 0
   age 00:00:02, expires in 24:00:00, 13716:20572 pkts,
713240:30853060 bytes, rule 7
   id: 4b47883900000033 creatorid: c576b1b2

By using this command you should be able to figure out which actual
rule, here rule 7, creates state for the active ftp data channel.

The same when the transfer has finished

all tcp 192.168.222.244:20 -> 192.168.222.20:64857       FIN_WAIT_2:FIN_WAIT_2
   [1427346950 + 65534] wscale 3  [259693437 + 66608] wscale 0
   age 00:00:59, expires in 00:00:38, 41209:61758 pkts,
2142876:92628227 bytes, rule 7
   id: 4b47883900000033 creatorid: c576b1b2

Happy rule/state hunting

Adriaan

all tcp 192.168.222.244:20 -> 192.168.222.20:64857       FIN_WAIT_2:FIN_WAIT_2
   [1427346950 + 65534] wscale 3  [259693437 + 66608] wscale 0
   age 00:00:59, expires in 00:00:38, 41209:61758 pkts,
2142876:92628227 bytes, rule 7
   id: 4b47883900000033 creatorid: c576b1b2