kern/56233: IPsec tunnel (ESP) over IPv6: MTU computation is wrong

[Thomas Pornin]

>Number:         56233
>Category:       kern
>Synopsis:       IPsec tunnel (ESP) over IPv6: MTU computation is wrong
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sun Aug 31 05:40:14 PDT 2003
>Closed-Date:
>Last-Modified:
>Originator:     Thomas Pornin
>Release:        FreeBSD 4.9-PRERELEASE alpha
>Organization:
>Environment:
System: FreeBSD gnah.bolet.org 4.9-PRERELEASE FreeBSD 4.9-PRERELEASE #0: Sat Aug 30 06:24:21 CEST 2003 root at gnah.bolet.org:/drive/obj/drive/src/sys/GNAH alpha

(problem actually occurs on i386 as well)


>Description:

The maximum ESP header size computation, function esp_hdrsiz(), in
sys/netinet6/esp_output.c, is wrong when used with a 16-byte block
cipher such as Rijndael. It is also wrong with a 8-byte block cipher,
but in the other direction, hence inducing no particular problem except
a possible slight performance hit.


>How-To-Repeat:

My office network and my own home network use the same ISP, which
provides IPv4 and IPv6 connectivity through ADSL links. Both networks
have a router/firewall running FreeBSD (4-STABLE from the end of August
2003).

I set up an IPsec tunnel between the two routers; the setkey
configuration file is simple:

spdadd [home-network-IP]/48 [office-network]/48 any -P out ipsec esp/tunnel/[home-router-IP]-[office-router-IP]/require ;
spdadd [office-network-IP]/48 [home-network]/48 any -P in ipsec esp/tunnel/[office-router-IP]-[home-router-IP]/require ;
add [home-router-IP] [office-router-IP] esp 0x10001 -m tunnel -E rijndael-cbc [symmetric-cipher-key] -A hmac-sha1 [authentication-key] ;
add [office-router-IP] [home-router-IP] esp 0x10002 -m tunnel -E rijndael-cbc [symmetric-cipher-key] -A hmac-sha1 [authentication-key] ;

(This is the configuration file on my home router; on the office router,
a similar configuration file is used.)

If I call N a machine of my home network, H my home router, O the office
router, and T a machine on the office network, I can send IPv6 packets
from N to T and back, and some tcpdumps show that those packet are duly
encrypted and authenticated between H and O.

Problems come when I try to send long packets. Typically, I establish
a TCP connection (through rlogin or ssh) and begin viewing a text with
an editor. The machine T wants to send a quite big packet to N. T has a
local MTU with O of 1500 (ethernet link) but the MTU is smaller between
O and H, so O sends to T an ICMP packet "too big" stating that T should
lower its MTU for this connection to 1407. Which T does. And there is
the true problem: 1407 is too big, the maximum being 1406. T and O enter
a loop where T repeatedly sends its 1407-byte packet, and O repeatedly
reject the packets and sends ICMPs stating that the MTU is 1407.


Details on the MTU computation:
External connectivity uses PPPoE and implies a MTU of 1492 (this is
important ! With a MTU of 1500, the problem is much less a nuisance).
The external IPv6 header uses 40 bytes. The ESP header uses 8 bytes
before the encrypted part, and 12 after (for the HMAC-SHA1 truncated
to 96 bits, as described in RFC 2404). So there remains 1432 bytes for
the encrypted part. The first 16 bytes are for the CBC initial value,
and there are 1416 bytes for data. However, since Rijndael uses 16-byte
blocks, the encrypted part must have a length multiple of 16. So the
real maximum encrypted data size is 1408 bytes. Since the Pad-Length
and Next-Header fields are mandatory, only 1406 bytes of data of
available.

Hence, the packets T sends to N through O and H must not exceed 1406
bytes, including their own IPv6 header.

>Fix:

The immediate work-around is to use an 8-byte block cipher such as
Blowfish. With such a block size, the MTU becomes 1422. In my setting, O
sends ICMP packets requesting a MTU of 1415, which is wrong again, but
in the safe direction: T sends packets shorter than needed, but data
flows.

A quick fix in the source code would be to change the code of
esp_hdrsiz() in sys/netinet6/esp_output.c, lines 139 and 153. This
function uses an estimate function which goes along the line:

ESP header length + IV length + 9 + Authlen

where the ESP header length is 8 bytes, the IV length is equal to the
cipher block length, and Authlen is the authentication algorithm
output length, here 12 for HMAC-SHA-1-96. The "9" is described in
a comment lines 149 and 150: it is the maximum padding length,
including the Pad-Length and Next-Header fields. This value is
correct for 8-byte block ciphers such as Blowfish and 3DES, but
should be "17" for 16-byte block ciphers.

So a quick fix would be to replace the "9" values in both lines 139
and 153 by "17".

A complete fix would require a more exact computation of the header
length, but it depends on the outter MTU and the cipher block length,
and I don't know to which extent that data is available at that
place in the kernel code.
>Release-Note:
>Audit-Trail:
>Unformatted: