svn commit: r277213 - in head: share/man/man9 sys/kern sys/ofed/include/linux sys/sys

[Adrian Chadd]

On 21 January 2015 at 06:00, Lawrence Stewart <lstewart at freebsd.org> wrote:
> On 01/20/15 09:22, Adrian Chadd wrote:
>> Yeah, it looks like you set c_cpu to timeout_cpu in
>> _callout_init_locked(), but then you only handle the case of the CPU
>> being changed in certain circumstances. You aren't handling the CPU
>> being initialised when the callout is first added.
>>
>> And, callout_restart_async() calls callout_lock(), which calls
>> CC_LOCK() on the callout CPU, which initially is zero. Then, it never
>> seems to be 'moved' into the correct CPU, even though it's being
>> called with a CPU id.
>>
>> So, if I am reading this all correctly, you aren't really handling
>> multi CPU callwheels at all. ;)
>>
>> In my instance, I'm seeing quite a lot of lock contention between the
>> userland threads, the network RX threads and the clock thread, all
>> contending on a single callout wheel being used for TCP timers. One of
>> the early goals for fixing up the RSS stuff in -HEAD was to make
>> per-CPU (Well, per-RSS-bucket really) TCP timers not contend with the
>> NIC RX processing path, and now it does. :(
>
> To clarify, you're seeing this with net.inet.tcp.per_cpu_timers=1?

Yup, RSS enables that by default.



-adrian