libpcap perf improvement? latest ideas?
sbahra at kerneled.org
sbahra at kerneled.org
Sat Oct 7 14:07:48 PDT 2006
> On Thu, 5 Oct 2006, Jochen Kaiser wrote:
>> after reading a german master thesis  (dated 12/2004) about pcap
>> performance (with comparison of linux and freebsd) I searched
>> freebsd resources for pcap improvements. Unofortunately I did not
>> find any improvements like PF_RING and/or efforts for reducing the
>> number of copy operations from device to user space.
I suggest you take a look at http://www.kerneled.org/?p=15
What is implemented here is reasonably light-weight but still uses a
ring buffer model. I agree that a reference model can be used to
reduce the number of copies done currently for BPF. I personally
haven't seen work there and I have yet the time unfortunately to
shut-up and just code such a thing.
>> Maybe I think too simple because I don't know how SMP fine locks
>> are influencing this (maybe it is very complex to improve that when
>> you want to avoid side effects.).
Basically, the reason for bad performance here in Linux in the
PF_PACKET sense is high system call overhead. BPF does collective
buffering, so system call over-head is less. Applications doing packet
analysis will also find a lot of required information in the BPF
header as well (such as timestamps). In PF_PACKET you are forced to do
a system call per-acquisition and another system for receiving the
time-stamp of the last packet read for example.
Other research papers have also shown that BPF still out-weights
PF_PACKET and other zero-copy PF_* solutions with small packets as
well. The kerneled.org post helps to explain this in a general sense.
Basically, zero-copy isn't the end-it-all solution but for larger
packets it makes great sense. The application I've been developing for
a while now makes use of both depending on average packet size.
On Thu, 6 Oct 2006, Robert Watson wrote:
> Quite a bit of work has been done on zero-copy for BPF, but none of it
> really commitable. Christian Peron (CC'd) and I have been talking
> about doing something that is commitable, but some of the details (such
> as memory ownership) are still very much up in the air.
Is there any archive or summary online noting the approaches you guys
would like to take with BPF? I am really interested in this, and when
my time permits would love to write some code. I am also interested in
the other work you noted that is not committable. Could we be provided
some URLs please? :-)
> PF_RING takes an interesting approach, and one we should look at, but we'd
> also like to keep all the benefits of BPF rather than discard them,
> so need to
> consider how best to apply elements of the approach in our context.
> I'd like to see something like this happen for FreeBSD 7.0, with a
> possible backport if it goes really well. :-)
The problem with PF_RING is it's static ring buffer model and concept
of marking slots. This doesn't make it so feasible for real-world
applications. I proposed a model that allows for dynamic ring buffer
size and signaling for soft and hard-limits to allow application
buffering to handle potential drops (or request specific allocation
strategies from the kernel). A Professor at my university also had an
idea of providing packet acquisition prioritization (push specific
packets to the beginning of the packet copy queue, which isn't
currently a concept in BPF) as a BPF language extension. An mmap
interface to BPF will be cool but unless we also provide stubs to
automize packet management from user-space it could be doomed to be
like PF_RING. :-P
Samy Al Bahra
More information about the freebsd-hackers