PF/FreeBSD 6 and FIN_WAIT2 TCP exhaustion

[Daniel Hartmeier]

On Mon, Jan 02, 2006 at 05:18:30PM +0100, TYBERGHIEN Eric TRANSPAC wrote:

> Can you help me to solve this feature. Is it a bug, a mechanism of DOS
> auto-protection or a mis-understood of the PF features ?

Look at the TCP RFC, sections "Knowing When to Keep Quiet" and "The TCP
Quiet Time Concept" starting on page 27 of

  http://www.faqs.org/rfcs/rfc793.html

After a client closes the connection to a server, it may not re-use the
same source port (to the same server port) before a quiet period has
passed. This is designed so packets from the first connection arriving
late at the server (due to taking different routes) can't disturb the
second connection.

This obviously limits the (sustained) rate at which your client can
connect to the server (to 65536 connections per 90 seconds, no matter
how fast your network may be). This was probably considered more than
enough when the TCP RFC was written, but nowadays people expect higher
connection rates in this case. The reasonable thing would be send
multiple HTTP requests over one persistent connection, since the
overhead of establishing (and tearing down) all those connections, for a
single request each, is significant. But yours is a benchmark, and not a
real application protocol, so I guess that's beside the point. :)

FreeBSD (and other OS) re-use ports from connections in TIME_WAIT state
when they need to. The assumption is that the disadvantage of not
detecting late arrivals of earlier connections is outweighed by the
increased connection rate possible.

You can tell pf to purge states in TIME_WAIT earlier, too. Those 90s are
merely the default,

  $ pfctl -st
  tcp.closed                   90s

and you can change it, either globally with 'set timeout tcp.closed 15'
or per rule with 'keep state (tcp.closed 15)'. Note that purging only
occurs in intervals (default 10s), so if you set the timeout to 15s, a
state may be purged after 15+10s. If you need higher resolution, lower
the interval (set timeout interval 5).

Daniel