[net] protecting interfaces from races between control and data ?

[Scott Long]

On Aug 7, 2013, at 2:00 PM, Andre Oppermann <andre at freebsd.org> wrote:

> On 07.08.2013 09:18, Luigi Rizzo wrote:
>> On Wed, Aug 7, 2013 at 5:26 AM, Mike Karels <mike at karels.net <mailto:mike at karels.net>> wrote:
>>    Jumping to (near) the end of the thread, I like most of Andre's proposal.
>>    Running with minimal locks at this layer is an admirable goal, and I agree
>>    with most of what was said.  I have a few observations on the general changes,
>>    or related issues:
>> 
>>    There was mention of taskqueues.  I think that with MSI-X, taskqueues
>>    should not be needed or used.  More specifically, having separate ithreads
>>    and taskqueues, with ithreads deferring to taskqueues after some limit, makes
>>    sense only for MSI and legacy interrupts.  With MSI-X, an interrupt thread
>>    should be able to process packets indefinitely with sufficient CPU resources,
>>    and there is no reason to context switch to a different thread periodically.
>>    A periodic "yield" might be reasonable, but if it is necessary, small packet
>>    performance will suffer.  However, most of this is internal to the driver.
>> 
>> 
>> i am not completely clear on what is the difference between ithreads and taskqueues.
> 
> The difference between ithreads and taskqueues is actually very small and mostly in
> name and how they're set up and kicked off when work is to be done.  Both are real
> kernel threads.  Most of the confusion, also in Mikes response, seems to come from
> the name ithreads for interrupt-threads.  However an ithread is not running in
> interrupt context, only the interrupt handler (called filter) does.  Scheduling a
> taskqueue from an ithread is a bit round-about but commonly done.  The bus_setup_intr(9)
> man page isn't helping to clear up the confusion.
> 

I think it might still be the case that threads are not allowed to sleep.  That means
no sx locks and no malloc(M_WAITOK), along with the obvious tsleep/msleep.  Taskqueues
have no such restriction.

An even rore relevant difference is that taskqueues have a much stronger
management API.  Ithreads can only be scheduled by generating a hardware interrupt,
can only be drained by calling bus_teardown_intr(), and cannot be paused.  Taskqueues
give you direct access to this control.

Scott