ipfw limit src-addr woes

[Ian Smith]

On Mon, 19 Feb 2007, admin wrote:
 > Ian Smith wrote:
 > > On Mon, 19 Feb 2007, admin wrote:
 > >  > Andre Santos wrote:
 > >  > > On 2/18/07, admin <admin at azuni.net> wrote:
 > >  > > 
 > >  > >> Hi, I'm trying to use ipfw's limit clause to limit the number of
 > >  > >> connections a single IP can have at the same time in a transparent
 > >  > >> web-proxy environment:
 > >  > >>
 > >  > >> 00350 skipto 401 tcp from x.x.x.x/x,y.y.y.y/y,z.z.z.z/z to any dst-port
 > >  > >> 80 in via if0 setup limit src-addr 10
 > >  > >> 00401 fwd local.ip.ad.dr,8080 tcp from x.x.x.x/x to any dst-port 80
 > >  > >> ... the rest fwd...
 > >  > >>
 > >  > >> as I understand the manpage, when the current number of connectiions is
 > >  > >> below 10, the action "skipto" is performed, else, the packet is dropped
 > >  > >> and the search terminates. But...
 > > 
 > > No, a packet is not dropped on a condition that fails a skipto test. 
 > > 
 > The manpage doesn't make this point clear.

You pretty much have to read it all .. several times .. a year.  One of
the things you note is that each rule is tested until a packet is either
allowed or denied by a rule, even until '65535 deny ip from any to any'.

 > limit {src-addr | src-port | dst-addr | dst-port} N
 >               The firewall will only allow N connections with the same 
 >               set of parameters as specified in the rule.

Yes, for this rule.  It still needs to be applied to an allow or deny
(or forward, divert etc, anything that terminates the search). 

 >       To limit the number of connections a user can open you can use the 
 > following type of rules:
 >             ipfw add allow tcp from my-net/24 to any setup limit src-addr 10
 >             ipfw add allow tcp from any to me setup limit src-addr 4

Yes.  Notice that these are allow rules, so the search terminates when
successfully matched.  It is assumed you'll later have rule/s denying
what you've not allowed.  True, this is not stated with every example. 
   
 > I'm assuming the packet gets silently dropped when the limit is 
 > overloaded but gets acted upon otherwise due to the stateful "limit" 
 > behaviour (keep-state in disguise). Just do a "skipto" when there's a 
 > state entry and that's it. And that's why the counter grows for 
 > established connections too, even though there's a "setup" modifier.

Can't tell without seeing your whole ruleset, but now that you know that
the skipto rule has NOT dropped the setup packets that don't match that
rule (including those exceeding the src-addr limit), I suspect you'll
find another rule has allowed them, on some other condition, later on.

 > skipto is a nice thing as it allows you to AND rules ;-)

or to OR, NAND or NOR :)  Reminds one that a ruleset is procedural.

 > Besides, that's what my humble testing came up with - connections over 
 > the limit DO get dropped... if done nicely.
 > 
 > As for the problem, it seems to me that all this noise is because of 
 > different timeouts in ipfw and TCP layer/whatever. The dynamic state 
 > entry for a connection expires while netstat -na still show the 
 > connection as ESTABLISHED, or, worse, the state entry is still there but 
 > the corresponding connection is in some half-closed state (FIN_WAIT_2, 
 > CLOSE_WAIT, LAST_ACK). The first case allows many more connections than 
 > "limit", while the second case won't let many good clients connect due 
 > to their buggy browsers not closing connections and letting the count 
 > build up. Could this be it?

I don't believe so.  They can only have been established in the first
place if the setup packet has been, somewhere in your ruleset, allowed.

Here it seems they're allowed (at least the ones from x.x.x.x/x) by the
fwd at 401 which has no 'setup' constraint, and will fwd both setup AND
established packets from x.x.x.x/x .. other rules, y and z, presumably.

Replaying .. trying not to do quite so much in one rule, but given you
can't just 'allow' here, since you want to run your fwd rules later: 

 > 00350 skipto 401 tcp from x.x.x.x/x,y.y.y.y/y,z.z.z.z/z to any dst-port \
 >       80 in via if0 setup limit src-addr 10

   00350 skipto 370 tcp from ${thatmob} to any dst-port 80 in via if0
   00360 skipto 401 ip from any to any   # bit clunky, but !(all that)

   00370 skipto 401 tcp from any to any setup limit src-addr 10  # goodies
   00380 deny tcp from any to any                           # else baddies

 > 00401 fwd local.ip.ad.dr,8080 tcp from x.x.x.x/x to any dst-port 80
 > ... the rest fwd...

FWIW: not only have I never used limit src-addr, but neither forward
with keep-state rules, so I could be talking ${hit} .. caveat bloggor.

Cheers, Ian