IPSEC disables TSO

[Andrew Gallatin]

Kip Macy writes:
 > On 8/16/07, Kip Macy <kip.macy at gmail.com> wrote:
 > > On 8/16/07, Andrew Gallatin <gallatin at cs.duke.edu> wrote:
 > > >
 > > > When tracking down an mxge 10GbE performance issue, I noticed that
 > > > simply compiling IPSEC into the kernel totally destroys 10GbE transmit
 > > > performance because it will silently disable TSO.
 > > >
 > > > The problem seems to be at least that the test on line 463 of
 > > > tcp_output.c (tp->t_inpcb->inp_sp == NULL) will always be
 > > > false.   If I comment that line out, TSO works as normal
 > > > (though I have no idea what would happen on an IPSEC connection).
 > > >
 > > > Is there some way to make this not happen?  I believe that the mxge
 > > > user simply wanted to run some handful of applications through ipsec
 > > > tunnels (perhaps even on a different NIC entirely).
 > >
 > > IPSEC encrypts the TCP header - how is the card going to do TSO?
 > 
 > 
 > I thought about this after sending and realized that the appropriate
 > response is that IPSEC needs to be careful to disable TSO when its in
 > use for a connection. What you're seeing was most likely done as an
 > expedient.

Yes, exactly, there needs to be a smarter test that can distingiush if
IPSEC is actually in use on a connection or not; I should have been
more clear about this.  The problem is that I have zero knowledge
about IPSEC, so I have no idea how to do this.

I'm worried that people will compile IPSEC into the kernel to run an
encrypted tunnel (or the TCP MD5 signature stuff for BGP), and
then be rather surprised that their their "normal" TCP performance
stinks.

Drew