Timing issue with Dummynet on high kernel timer interrupt

[Hans Petter Selasky]

On 11/06/15 11:08, Luigi Rizzo wrote:
> On Fri, Nov 6, 2015 at 10:52 AM, Hans Petter Selasky <hps at selasky.org> wrote:
>> On 11/06/15 09:50, Luigi Rizzo wrote:
>>>
>>> On Fri, Nov 6, 2015 at 9:44 AM, Hans Petter Selasky <hps at selasky.org>
>>> wrote:
> ...
>>>> Hi,
>>>>
>>>> The C_DIRECT_EXEC flag reduces task switching overhead, that you don't
>>>> have
>>>> to wakeup a thread to wakeup the dummynet worker thread. It affects
>>>> timing.
>>>
>>>
>>> Hans,
>>> thanks for the explanation.
>>>
>>> Can you clarify the behaviour of C_DIRECT_EXEC ?
>>> Does this mean that the task is run within some common
>>> thread instead of a dedicated one ?
>>
>>
>> Hi Luigi,
>>
>> C_DIRECT_EXEC means that the timer callback is executed directly from the
>> fast interrupt filter of the timer or IPI.
>>
>>>
>>> If so, for this type of task (dummynet may run at high rate
>>> and use a significant amount of cpu time) it may be a good
>>> idea to remove C_DIRECT_EXEC altogether.
>>
>>
>> The ipfw dummynet code is not run from the timer callback. It is run from a
>> taskqueue. I suspect there is likely a bug somewhere. At the moment it is
>> not clear to me if there is a bug in the callout subsystem, that the when
>> the timer is started with 1 tick delay it doesn't kick in until after 50ms
>> or so at HZ=4000. Or if the dummynet's task is doing a lot of work for 50ms.
>> I think we need some more information to nail this one.
>
> It certainly does not run for 50ms, but it might occasionally keep the
> thread busy for some 10-50us (I doubt it is longer than that)
> and possibly cause the reschedule request to
> fall into the interval where it should actually run.
>
> So if your theory is correct, it may well be that the callout system
> sees the request "in the past" (possibly as a result as some incorrect
> wraparound, or undefined behaviour on integer wraps) and then the
> event is only recovered when the callout wheel (or whatever is the
> underlying implementation) happens to go again through the entry.
>
> What is so magic in the values we see (400 or 600 or 40ms) i have no idea.
>

Rasool:

It might be worth trying to set:

kern.eventtimer.periodic=1

In /boot/loader.conf . Can you test that too?

You need to reboot before the setting takes into effect.

Luigi:

I'm wondering if there is a problem with:

cpu_new_callout(a,b,c);

--HPS