Multiple TX queue support in nic driver

[K. Macy]

(added -drivers back to the CC for your mail's educational value)

On Tue, Sep 20, 2011 at 5:02 PM, Naresh <gbal.naresh at gmail.com> wrote:
> Hi Macy,
>
> Thanks a lot for the detailed explanation, Now I understand why the ring buffer is used.
>
> In Linux all these things are maintained in the stack, is there any advantage of FreeBSD moving these things in to drivers?

I doubt it. When I ported Chelsio's T3 10G Linux driver to FreeBSD
(cxgb) FreeBSD had no support for multiple transmit queues. At that
time all packets were relayed to the driver by a per-driver instance
linked list (IFQ) and then calling ifp->if_start(ifp). After some
discussion with others I extended the API with if_transmit to allow
the caller to do direct transmit if possible or defer transmission in
a driver specific fashion. By adding buf_ring I provided drivers with
a much faster queueing mechanism without enforcing any sort of policy.
At the time I hadn't done much work in the network stack beyond
updating some of the wireless drivers so my only real objective was
getting the network stack out of the way of the driver. At this point
we've established a sufficiently consistent driver programming model
with if_transmit that it would make sense to move a lot of that out of
the driver and in to the stack.


> During module attach, Linux drivers report number of TX queues they support and based on that value stack creates that many software queues. These allocated queues reference is stored in netdev structure, so that they can be accessible by both driver and stack. In XMIT routine, skbuff indicates which queue to use and based on that value packet is transmitted on appropriate hardware TX queue.
>
> Both the approaches of the OS's are similar, Only difference is in Linux these are taken care by stack.

I think that is the right approach, it simply hasn't reached the front
of anyone's priority queue. For my non-work hours I'm busy with
fleshing out my user level network stack, and the others who might do
the work are themselves busier still.

> Once again thanks a lot for you help.
>

Cheers