Rewritten TCP reassembly
andre at freebsd.org
Mon Dec 13 13:34:21 PST 2004
Andrew Gallatin wrote:
> Andre Oppermann writes:
> > I've got some excellent review feedback from Mike Spengler and he found
> > a off-by-one queue limit tracking error.
> > http://www.nrg4u.com/freebsd/tcp_reass-20041213.patch
> Here are the same tests running your new patch in comparison to a
> stock 6.x kernel from Friday. It looks like everything is faster.
> I've never seen 3.7Gb/sec on FreeBSD before. Nice work!
> The "before" and "tcp_reass-20041213" kernels differ only in the
> contents of the above patch. I ran netperf with 20 times for each of
> 4 socket buffer sizes (64KB, 128KB, 256KB, and 1MB) before and after
> patching. All tests were run with net.isr.enable=1, and
> machdep.cpu_idle_hlt=0. CPU was pretty much maxed thoughout. (SMP
> kernel, 1 HTT p4). All the numbers in Mb/sec. The sender was running
> linux-2.6.6 (also SMP on an identical HTT P4).
> The out-of-order accounting bug seems to be gone.
Perfect. I want to get this patch into the tree next.
I have already the next round in the works which is optimized even more
by merging consecutive mbuf chains together (at the moment I have packet
segment chains which have a direct pointer to the mbuf at the end of the
chain) and which get passed in one go to soappend_stream. This removes
the "present" loop and simplifies the general code a bit more again.
With this and two other optimizations I have in mind you should be able
to get very close to the theoretical maximum bandwidth of your current
4Gig Myrinet cards.
There are a couple of other TCP tweaks that would help your special case
some more now though.
More information about the freebsd-net