MQ Patch.

[Andre Oppermann]

On 06.11.2013 16:01, Randall Stewart wrote:
> The other problem with no software ring is you will
> have some rough time doing AQM approaches such as Codel.

The idea is to have different modes: when no QoS is configured the packets
are going straight into the deep DMA rings.  When QoS is configured the
queue scheduler or discipline manages a software queue and can select a
hardware DMA ring depth.  The depth is managed by admitting only a certain
amount of packets/data and the hardware doesn't have to shorten the ring
itself.  The depth probably is defined in microseconds at actual link speed.
It won't be perfect and deviate by a few percent to keep the calculations
simple (IFG, pre/post-amble, etc).

With this the QoS, if enabled, can set the DMA depth to 1000us (1ms) for
example to manage its software queues and overall effectiveness.  Or it
chooses 200us (0.2ms).  It does have to balance the available resources
though to prevent pipeline bubbling from scheduling latency.

> Codel does a head drop, if there is no software ring you must get
> the hardware to implement the head-drop.

If you enable Codel queuing would happen in software and with a rather
shallow DMA ring to make it effective.

-- 
Andre

> Pi on the other hand does tail drop, which is why Cisco is pushing it, since
> its much more friendly for both software and hardware queues combined (which
> is what Cisco has).
>
> Of course Pi has Cisco you can't sue me IPR with it thus the need for a plugin, but
> Codel from Van does not have this.. but has that little niggle that it only
> works with a queue in software that you can head-drop from .
>
> R
> On Oct 29, 2013, at 2:02 PM, Adrian Chadd wrote:
>
>> [snip everything]
>>
>> ok, I've reviewed the work.
>>
>> TL;DR - it's a clearly correct step in the right direction, but I
>> think we need to just think it through a tad bit more first.
>>
>> There have been queue discipline and queue management discussions in
>> the past. Randall's work is a good step in that direction.
>>
>> I think though that we can take a step back up a little further.
>>
>> * In terms of queuing frames into multiple queues - yes, we absolutely
>> should have an if_transmit() path to the driver that obeys "a" QoS
>> field in the mbuf and pushes it into the relevant queue - with
>> randalls work, it's in the driver, but it doesn't _have_ to be;
>> * In terms of queue servicing and management - we likely need to have
>> a variety of queue plugins that determine which frame from which queue
>> gets chosen next to hand to the hardware. The hardware may have
>> multiple queues! The hardware may have one queue! The application
>> developer may only want to use one queue! That should be flexible and
>> easy to plug into things.
>> * Then we need to support dropping frames during queue and dropping
>> frames during dequeue (ie, on its way to the hardware). That way we
>> can implement the currently interesting kinds of queue disciplines (eg
>> CODEL, etc.)
>> * Should this be done at the driver layer (ie it's a library that each
>> driver creates and owns), or as a layer above it, controlling the
>> network device (ie, the linux queue discipline method.)
>>
>> So, my comments:
>>
>> * I don't like how it's hard-coding drbr's into the drivers. Yes, the
>> underlying state should be a drbr for now. But I'd rather we have a
>> queue discipline plugin API that drivers create an instance of.
>> * It'll have methods to init/flush the rings, queue a frame into a
>> ring, dequeue a frame from a ring, be notified of transmit completions
>> so more work can be done, etc.
>> * For people who do latency-sensitive things, they can just bypass
>> this entirely and go straight to the hardware queues without going
>> through this kind of intermediary queue layer.
>>
>> Randall - I think we can take your work and turn it into a net library
>> that implements your queue management routines. That way we can start
>> enabling people to tinker with it and replace it if they need to.
>>
>> What do you think?
>> _______________________________________________
>> freebsd-net at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>
>
> ------------------------------
> Randall Stewart
> 803-317-4952 (cell)
>
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>
>