kern/103135: ipsec with ipfw divert (not NAT) encodes a packet twice breaking PMTUD

Eugene Grosbein eugen at
Mon Sep 11 15:40:29 PDT 2006

The following reply was made to PR kern/103135; it has been noted by GNATS.

From: Eugene Grosbein <eugen at>
To: FreeBSD-gnats-submit at
Cc: net at
Subject: kern/103135: ipsec with ipfw divert (not NAT) encodes a packet
	twice breaking PMTUD
Date: Mon, 11 Sep 2006 21:41:49 +0800 (KRAST)

 >Number:         103135
 >Category:       kern
 >Synopsis:       ipsec with ipfw divert (not NAT) encodes a packet twice breaking PMTUD
 >Confidential:   no
 >Severity:       serious
 >Priority:       high
 >Responsible:    freebsd-bugs
 >State:          open
 >Class:          sw-bug
 >Submitter-Id:   current-users
 >Arrival-Date:   Mon Sep 11 13:50:16 GMT 2006
 >Originator:     Eugene Grosbein
 >Release:        FreeBSD 6.1-STABLE i386
 Svyaz Service JSC
 System: FreeBSD 6.1-STABLE FreeBSD 6.1-STABLE #1: Thu Sep 7 13:31:53 KRAST 2006 root at i386
 	options IPDIVERT
 	options IPSEC
 	options IPSEC_ESP
 	When outgoing packet encoded due to corresponding IPSEC policy
 	is passed to divert socket (f.e. to ipacctd for accounting),
 	it is encoded second time with IPSEC then. Besides obvious
 	logic error, this also results in broken Path MTU Discovery.
 	Use a kernel with options IPDIVERT, IPSEC, IPSEC_ESP
 	(my kernel also contains IPSEC_FILTERGIF, but this should not matter).
 	Suppose there are two local nets numbered
 	and, each has a FreeBSD router
 	( and Routers make gif(4) tunnel between
 	and use IPSEC transport mode to encrypt its contents.
 	Their external IP addresses are and
 	Here is /etc/ipsec.conf:
 add esp 1007 -m transport -E blowfish-cbc "xxx";
 add esp 2007 -m transport -E blowfish-cbc "yyy";
 spdadd any -P in  ipsec esp/transport/;
 spdadd any -P out ipsec esp/transport/;
 	Another router has similar /etc/ipsec.conf.
 	Use this script to prepare IPSEC keys for tcpdump:
 setkey -D | awk '
   /^[1-9]/  { ip=$2; }
   $1=="esp" { 
 		sub(/spi=[^(]+\(/, "", $3);
 		sub(/\)/, "", $3);
 		printf"%s@%s ",$3,ip;
   $1=="E:"  { printf "%s:0x%s%s%s%s\n", $2, $3, $4, $5, $6; }
 ' > /tmp/keys.txt
 	When we use one of the routers to run
 	"tcpdump -s0 -n -p -i $iface -E /tmp/keys.txt esp",
 	we see that traffic is encoded with ESP and decoded properly,
 	still good.
 	Now install ports/net-mgmt/ipacctd, run it manually with
 	"ipacctd -p 4000", then command:
 	"ipfw add 10 divert 4000 ip from any to any out"
 	and rerun tcpdump command shown above. You'll see that
 	outgoing packets are encapsulated with ESP twice.
 	The same you'll see at the other side for incoming packets.
 	Unknown. The only known workaround is to avoid diverting ESP.
 	This workaroung is not always acceptable.
 Eugene Grosbein
 freebsd-bugs at mailing list
 To unsubscribe, send any mail to "freebsd-bugs-unsubscribe at"

More information about the freebsd-bugs mailing list