TCP loopback socket fusing

[Andre Oppermann]

When a TCP connection via loopback back to localhost is made the whole
send, segmentation and receive path (with larger packets though) is still
executed.  This has some considerable overhead.

To short-circuit the send and receive sockets on localhost TCP connections
I've made a proof-of-concept patch that directly places the data in the
other side's socket buffer without doing any packetization and other protocol
overhead (like UNIX domain sockets).  The connections setup (SYN, SYN-ACK,
ACK) and shutdown are still handled by normal TCP segments via loopback so
that firewalling stills works.  The actual payload data during the session
won't be seen and the sequence numbers don't move other than for SYN and FIN.
The sequence are remain valid though.  Obviously tcpdump won't see any data
transfers either if the connection has fused sockets.

Preliminary testing (with WITNESS and INVARIANTS enabled) has shown stable
operation and a rough doubling of the throughput on loopback connections.
I've tested most socket teardown cases and it behaves fine.  I'm not entirely
sure I've got all possible path's but the way it is integrated should properly
defuse the sockets in all situations.

Testers and feedback wanted:

  http://people.freebsd.org/~andre/tcp_loopfuse-20100913.diff

-- 
Andre