svn commit: r277213 - in head: share/man/man9 sys/kern sys/ofed/include/linux sys/sys

[Adrian Chadd]

On 19 January 2015 at 21:20, Hans Petter Selasky <hps at selasky.org> wrote:
> On 01/20/15 06:04, Adrian Chadd wrote:
>>
>> On 19 January 2015 at 20:30, Hans Petter Selasky <hps at selasky.org> wrote:
>>>
>>> On 01/19/15 22:59, Adrian Chadd wrote:
>>>>
>>>>
>>>> Hi,
>>>>
>>>> Would you please check what the results of this are with CPU specific
>>>> callwheels?
>>>>
>>>> I'm doing some 10+ gig traffic testing on -HEAD with RSS enabled (on
>>>> ixgbe) and with this setup, the per-CPU TCP callwheel stuff is
>>>> enabled. But all the callwheels are now back on clock(0) and so is the
>>>> lock contention. :(
>>>>
>>>> Thanks,
>>>>
>>>
>>> Hi,
>>>
>>> Like stated in the manual page, callout_reset_curcpu/on() does not work
>>> with
>>> MPSAFE callouts any more!
>>
>>
>> Hm!
>>
>
> Hi Adrian,
>
>> How many places in the kernel did you leave like this? :P
>
>
> :-)
>
>>
>> I mean, I'm glad to have stuff be forced to be cleaned up, but you
>> didn't even leave a KASSERT or a debug warning that something
>> unsupported is being done. I'm sure I'm not going to be the first
>> person to be caught out like this.
>
>
> MPSAFE is still valid and fully useable and can be used with
> callout_reset_curcpu/on(), but the callout CPU will remain at zero.
> There is no need for a KASSERT() yet.

Right, but people won't know that their callout won't be scheduled on
the CPU they've asked for and there's no way to get notified of that.
So something debug-y may be useful, even to make it easy to track down
situations where sometimes it succeeds and sometimes it fails (because
well, there's lots of places in the kernel where locking is ..
suboptimal.)


>>
>>> You need to use callout_init_{mtx,rm,rw} and remove the custom locking
>>> inside the callback in the TCP stack to get it working like before!
>>
>>
>> Would you please give me a hand with this? I've sunk a lot of (unpaid,
>> personal) spare time into getting the RSS stuff into shape and now a
>> lot of it just plainly doesn't do anything. :(
>
>
> I'll send you a patch in an hours time from now for 11-current. This should
> be fairly trivial and then you can test and review it!

Sweet, thanks. I'l test it, but anything that changes the locking to
TCP is going to need a more thorough review. The "there be dragons"
disclaimer is appropriate. :)


-adrian