Aggregating many ports into one for tcpdump server. (also sampling before libpcap)

[Peter Wood]

Morning,

 >>> Looking thru the archives, it seems ng_one2many (in this case
 >>> 'many2one') is what I am looking for.  Am I barking the right tree 
here?

Strangely enough this is the exact situation I was looking into on 
Friday for two mirror ports from our border routers via aggregation 
switches.

I had seen the netgraph solution however I had initially ignored 
if_bridge as I don't want the packets to be sent to the opposing devices.

 >> I've had several reports of significantly improved packet capture
 >> rates at high speeds with it, but it's not yet in the tree because we
 >> feel it needs more evaluation and review.  I hope to ship some form of
 >> zero-copy BPF buffer support in FreeBSD 8, and possibly even MFC it.
 >> Any feedback you might have would be most helpful.

As I am about to reinstall the server in question, I too shall give the 
zero copy code a go and report back. For reference on our two links the 
mirrored data is fed into snort (as well as tcpdump for "interactive" 
investigation) at about 700mbs average.

Roberts suggestion of a 10Gbe interface hits home for me as we're in the 
middle of planning (or should I say plotting) an upgrade to our 
connection to the UK academic network to 10Gbe (although at maximum of 
2.5Gbs due to our RENs connection, we're working on that too ;).

At which point we might have to consider using sampling, unfortunately 
the aggregation switch we use doesn't support sampling on a mirror port.

I know it's a tad off topic, but having a quick look that's not 
something I see libpcap shouting about. After very quick thinking would 
that have to be implemented in the kernel before the packets where 
passed to BPF?

I'd prefer to use sampling rather then just accepting kernel droped 
packets to ensure fair selection over a time period, rather then only 
collecting the start of that period and then nothing else.

I'd be willing to look into implementing that perhaps in the same way 
that Juniper Networks do for their sampling, ie. a maximum number of 
packets to be sampled in a second, how often to sample in terms of 
packets and then when sampling how many packets it should sample.

Cheers,

Peter Wood
Network Security Specialist
Information Systems Services
Lancaster University
-- 
Peter Wood <peter at alastria.net>