TCP Reassembly Issues

[Kris Bauer]

On Thu, Nov 24, 2011 at 10:20 PM, Lawrence Stewart <lstewart at freebsd.org>wrote:

> On 11/25/11 14:19, Kris Bauer wrote:
>
>> On Thu, Nov 24, 2011 at 8:00 PM, Jeremy Chadwick
>> <freebsd at jdc.parodius.com>wrote:
>>
>>  On Thu, Nov 24, 2011 at 07:13:39PM -0600, Kris Bauer wrote:
>>>
>>>> On Thu, Nov 24, 2011 at 5:35 PM, Adrian Chadd<adrian at freebsd.org>
>>>>
>>> wrote:
>>>
>>>>
>>>>  Have you tried disabling the tcp offload features of your NIC?
>>>>>
>>>>>
>>>>> Adrian
>>>>>
>>>>>
>>>> To test this, I added net.inet.tcp.tso=0 to sysctl.conf and restarted
>>>> the
>>>> box; it didn't work.  net.inet.tcp.reass.cursegments immediately started
>>>> climbing up and were exhausted within an hour.
>>>>
>>>
>>> I think Adrian was referring to RXCSUM and TXCSUM on your NIC; TSO is
>>> another offloading feature.
>>>
>>> See ifconfig(8) for how to disable those.
>>>
>>> Be aware that disabling them in real-time (e.g. ifconfig xxx -rxcsum
>>> -txcsum) may cause problems; there are some NIC drivers on FreeBSD which
>>> do not like you doing this once the NIC has established link (meaning
>>> "reloading the driver" (for lack of better term) results in wonky
>>> behaviour).  So you may instead want to add those hyphen-options to your
>>> ifconfig_XXX lines in /etc/rc.conf and reboot the box.
>>>
>>> If none of this solves the problem, then I consider this a priority 0
>>> blocker (read: "all hands on deck") issue with the IP stack in FreeBSD
>>> 9.x and will need immediate attention.
>>>
>>> I would strongly recommend a developer or clueful end-user begin
>>> tracking down who committed all of these bits and CC them into the
>>> thread.  I would start by looking who implemented the
>>> net.inet.tcp.reass.cursegments sysctl, because that isn't in RELENG_8 at
>>> all.
>>>
>>>
>> I have added -rxcsum -txcsum -tso to rc.conf and rebooted the box.  This
>> has not solved the problem.  After a half-hour usage, I'm already up to
>> reass.cursegments=2182 and it keeps climbing.
>>
>
> This is pretty much guaranteed to be an accounting problem in the TCP
> reassembly code (netinet/tcp_reass.c), not a driver related issue. I would
> not expect any amount of tweaking, tuning or driver option twiddling to
> change the outcome (but if you do find something which alleviates it, do
> let us know).
>
> Kris, are you in a position to test kernel patches on the machine which is
> experiencing this problem?
>
> Cheers,
> Lawrence
>


I'd be happy to test kernel patches with this machine.

Thanks,
Kris