RFC: revising netmap ring initialization

[Luigi Rizzo]

I would like people's suggestions on the following
​ topic.​

Right now, on enteringng netmap mode on a NIC we do an ifconfig
down, flush the tx and rx queues, and replace the rx buffers with
the netmap ones.  Similarly, on exit, we down the interface, flush
queues and restore the mbufs/skbufs.  The annoying side effect is
that in this way the link goes down and sometimes it takes a long
time for autonegotiation and/or spanning tree to restore connectivity.

I was thinking of a different way, as follows (i omit the locking
requirements):

1. always keep the interface active and the mbufs associated to the
NIC rings as allocated by the network stack, whether or not the
interface is in normal or netmap mode;

2. when entering netmap mode just record the new operating mode (in
turn redirecting the interrupt handlers to use the netmap routines
rather than the usual *txeof(), *rxeof() ), and set the ring indexes
according to the state of the ring (ie. pending tx mbufs are reported
as unavailable);

3. the *_rxsync() routine in each driver will track which slots are
still using mbufs (initially, all of them). When an incoming packet
is in an mbuf, *_rxsync() will copy the payload in the netmap buffer,
and mark the slot as a standard netmap bufffer for the future.
After one round through the ring, all buffers will be standard
netmap and there is no copy anymore.

4. on the tx side things are even simpler, all it takes is to
do an m_freem() of completed tx mbufs and then mark the slots
as available. Here again, after one round there is no overhead
anymore.

5. when switching out of netmap mode things are a bit trickier,
because we cannot release immediately the netmap buffers that are
under processing, so we should add special code in the
standard *txeof()/*rxeof() to report when the netmap buffers
are not in use anymore. The code changes in the standard
driver should be relatively simple, but the annoying thing is
that we cannot free the netmap buffers on detach.

For the tx side we could just loop shortly until
the tx queue has been drained., which should happen quickly.
For the rx side, however, we cannot tell when we will receive
incoming traffic and reclaiming a buffer that is already in
the rx engine may require a ring reset.

So, any comment on the above, especially on the last issue ?

cheers
luigi