Blocking very many (tens of thousands) ip addresses in ipfw

[Pieter de Goeje]

On Wednesday 14 January 2009 17:23:25 Artem Kuchin wrote:
> I need to block around 150000 ip addreses from acccess the server at all
> at any port.  The addesses are random, they are not nets.
> These are the spammer i want to block for 24 hours.
> The list is dynamically generated and regenerated every hour or so.
> What is the most efficient way to do it?
> At first i thought doing ipfw rules using 5 ips per rule, that would
> result in 30000 rules! This will be too slow!
> I need to something really quick and smart. Like matching the first
> number from ip (195 from 192.1.2.3),
> if it does not match - skip, if it does - compare the next one
> and so on.

Quoting ipfw(8):
LOOKUP TABLES
     Lookup tables are useful to handle large sparse address sets, typically
     from a hundred to several thousands of entries.  There may be up to 128
     different lookup tables, numbered 0 to 127.

net.inet.ip.fw.dyn_buckets should probably also be increased to efficiently 
handle 150k IPs.

-- 
Pieter de Goeje