TCP Reassembly Issues

[Rick Macklem]

George Mitchell wrote:
> On 11/24/11 21:00, Jeremy Chadwick wrote:
> >[...]
> > If none of this solves the problem, then I consider this a priority
> > 0
> > blocker (read: "all hands on deck") issue with the IP stack in
> > FreeBSD
> > 9.x and will need immediate attention.
> >
> > I would strongly recommend a developer or clueful end-user begin
> > tracking down who committed all of these bits and CC them into the
> > thread. I would start by looking who implemented the
> > net.inet.tcp.reass.cursegments sysctl, because that isn't in
> > RELENG_8 at
> > all.
> >
> 
> I've tried out the 9.0 release candidates, and what I notice is that
> for
> a few minutes after the system starts, I get wonderful NFS read
> throughput (7+ MB/s over a 100 megabit interface) -- more than twice
> as
> fast as 7.n or 8.n on the same hardware -- quickly degrading to
> abysmal
> (less than 0.5 MB/s). Is this possibly related to the problem under
> discussion? -- George Mitchell
> 
Well, when I've seen NFS perf. degrade like this, it has usually been
related to RPC transport (and TCP is the default for 9.0).

Just from reading some of the thread, it sounds like this problem will
result in the FAIL count (the last #) for "vmstat -z" for tcpreass will
increase and/or net.inet.tcp.reass.cursegments increases to
net.inet.tcp.reass.maxsegments.

I'd suggest that, after the NFS perf has degrades, you:
# vmstat -z | fgrep tcpreass
- and see how big the last # is
# sysctl -a | fgrep reass
- and see how cursegments compares with maxsegments

If these don't indicate that is the TCP Reassembly Issue, then...

There are many other possibilities w.r.t. the NFS perf. degradation.
Most often I've seen it when the net interface hardware/device driver
starts dropping packets (like happens on this laptop with an el-cheapo
re net interface in it).

You can capture a packet trace after the performance has degraded with
tcpdump and look to see if TCP segments are being lost/retransmitted.
(Although wireshark knows NFS and is nice for this, because it shows
 relative sequence numbers, the TCP dump will show you the TCP level
 retries, etc.)

Good luck with it, rick

> P.S. A lot of other 9.0 features look very nice indeed!
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to
> "freebsd-stable-unsubscribe at freebsd.org"