82576 + NETMAP + VLAN

Giuseppe Lettieri g.lettieri at iet.unipi.it
Thu Feb 11 09:06:07 UTC 2016

Il 10/02/2016 14:53, Slawa Olhovchenkov ha scritto:
> On Wed, Feb 10, 2016 at 02:33:20PM +0100, Giuseppe Lettieri wrote:
>> Il 10/02/2016 12:59, Slawa Olhovchenkov ha scritto:
>>> Can you look also on second issue?
>>> PS: What need from me? May be open PR?
>> May you provide some example code that triggers the issue?
> This is about 700 lines of code (not very clear), may be I can describe it?

I just need some code to trigger the problem locally. Don't worry about 
the clarity and the line count, unless you cannot share the code for 
other reasons.


> First: this is thread code. I think it impotant point.
> All sync with netmap go throw kevent() interface.
> Main thread got information about two NIC (determine number of rings)
> and spawn 1 balancer thread and NCPU worker thread.
> All thread pinned to cores.
> Balancer thread open all rings on both NIC separate (i.e. open 6+6
> NETMAP fds for hardware rings) and allocate on first NIC and open
> NCPU*2 master NETMAP pipes.
> All fds register in kevent for read event as
> EV_SET(kp, fd, EVFILT_READ, EV_ADD | EV_ENABLE | EV_CLEAR, 0, 0, kidx);
> and for future write event as
> EV_SET(kp, fd, EVFILT_WRITE, EV_ADD  | EV_DISABLE, 0, 0, 0);
> after write occur next event will be added:
> EV_SET(kp, fd, EVFILT_WRITE, EV_ADD | EV_DISPATCH, 0, 0, 0);
> Each worker thread open two slave NETMAP pipes in same maner as above.
> Balancer got packet from inside hardware ring, dispatch (zero-copy) to inside master pipe.
> Worker got packet from inside slave pipe and zero-copy to outside slave pipe.
> Balancer got  packet from outside master pipe and dispatch (zero-copy) to outside hardware ring.
> And vise versa.
> After program start I have deadlock instantly (with WITNESS enabled kernel).
> W/o WITNESS deadloc occur only under load.
> W/o EV_DISABLE/EV_DISPATCH NETAMP don't see output packets.
> NETMAP don't forward incoming packet to program when open and attach to NETMAP do in main thread.

Dr. Ing. Giuseppe Lettieri
Dipartimento di Ingegneria della Informazione
Universita' di Pisa
Largo Lucio Lazzarino 1, 56122 Pisa - Italy
Ph. : (+39) 050-2217.649 (direct) .599 (switch)
Fax : (+39) 050-2217.600
e-mail: g.lettieri at iet.unipi.it

More information about the freebsd-stable mailing list