pf synproxy

[Daniel Hartmeier]

On Wed, Jul 28, 2010 at 01:04:42PM -0700, Justin wrote:

> Logged to files and dumped;
> 
> Outside:
> 19:58:09.571810 IP (tos 0x0, ttl 118, id 12726, offset 0, flags [DF], 
> proto TCP
> (6), length 52)
>     REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [S], cksum 0xb15f 
> (correct), se
> q 3641874856, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], 
> length 0
> 19:58:09.571835 IP (tos 0x10, ttl 64, id 6840, offset 0, flags [DF], 
> proto TCP (
> 6), length 44)
>     TARGET_HOST.80 > REMOTE_CLIENT.56270: Flags [S.], cksum 0xa352 
> (correct), s
> eq 4110180879, ack 3641874857, win 0, options [mss 1460], length 0
> 19:58:09.583132 IP (tos 0x0, ttl 118, id 12727, offset 0, flags [DF], 
> proto TCP
> (6), length 40)
>     REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [.], cksum 0xc01e 
> (correct), ac
> k 4110180880, win 64240, length 0

The handshake between client and synproxy looks fine.

Now the synproxy should handshake with the server:

> Inside:
> 19:58:09.583144 IP (tos 0x10, ttl 64, id 6841, offset 0, flags [DF], 
> proto TCP (
> 6), length 44)
>     REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [S], cksum 0x2a53 
> (correct), se
> q 2026752522, win 0, options [mss 1460], length 0
> 19:58:09.593505 IP (tos 0x10, ttl 64, id 6842, offset 0, flags [DF], 
> proto TCP (
> 6), length 44)
>     REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [S], cksum 0x2a53 
> (correct), se
> q 2026752522, win 0, options [mss 1460], length 0

The synproxy keeps sending its SYN, but never gets a SYN+ACK from
the server. The handshake doesn't complete.

So, the question is why the server doesn't reply with SYN+ACK.

- Does the server receive the SYN? Try tcpdump on the server itself.
- Does the server consider it invalid for some reason? Firewalling
  on the server? netstat -s tcp on the server (any errors increasing)?
  Maybe it doesn't like the zero sized window? What OS is it?
- The server hasn't seen the client's original SYN, has it, i.e. through
  some other path, bypassing the pf box? tcpdump on the server to be
  sure...
- Does the server actually reply with SYN+ACK? Again tcpdump on the
  server itself.
- If so, why does that SYN+ACK not reach the synproxy?
  - Default gateway of the server correctly set to synproxy's address?
  - netstat -anr, arp -a on the server?

>    Some people on the OpenBSD list suggested it had something to do
> with synproxy only working on the interface that has a default gateway -
> I've tried all variations of em0/1 being the default route out, still no
> changes.

When synproxy generates a packet (either the SYN+ACK towards the client,
or the SYN and ACK towards the server), it will send those out with
ip_output(), which simply uses the routing table to decide which
interface to use to send the packet. Usually, like in your case, the
client is reachable through the default gateway and the server through
a local network route, and things work fine. There just isn't any
additional logic in synproxy to route generated packets differently
(as in "route SYN+ACK always back out through the interface the client
SYN came back in through", or such, which some people seem to assume),
and route-to/reply-to do not apply to synproxy generated packets, IIRC.

Daniel