Large TCP send socket buffer optimizations

[Andre Oppermann]

With the TCP socket buffer autosizing and generally larger socket buffers
for high bandwidth and high delay connections tcp_output() has become
increasingly inefficient for sending segments.  For every segment sent it
is traversing the entire socket buffer mbuf chain until it finds the offset
to continue from.  Usually this is close to the end of the chain.  Once it
got past a few dozen mbufs it starts to bust the CPU caches and performance
starts to fall off.

This patch solves the problem by maintaining an offset pointer in the socket
buffer to give tcp_output() the closest mbuf right away avoiding the traversal
from the beginning.

With this patch we should be able to compete nicely for the Internet land
speed record again.

The patch is here:

  http://people.freebsd.org/~andre/sockbuf_sndptr-20070301.diff

Any testing, especially on 10Gig cards, and feedback appreciated.

-- 
Andre