kern/119036: [patch] enc(4) and dummynet together produce kernel panics

[Manuel Kasper]

>Number:         119036
>Category:       kern
>Synopsis:       [patch] enc(4) and dummynet together produce kernel panics
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Dec 26 10:30:01 UTC 2007
>Closed-Date:
>Last-Modified:
>Originator:     Manuel Kasper
>Release:        6.2-RELEASE-p9
>Organization:
>Environment:
FreeBSD fb62.neon1.net 6.2-RELEASE-p9 FreeBSD 6.2-RELEASE-p9 #0: Sun Dec 23 15:09:45 CET 2007     root at fb62.neon1.net:/usr/src/sys/i386/compile/GENERIC_IPSEC_ENC  i386

>Description:
When enc(4) is used in conjunction with ipfw+dummynet, certain configurations may lead to a kernel panic like the following:

Fatal trap 12: page fault while in kernel mode
fault virtual address   = 0x4
fault code              = supervisor write, page not present
instruction pointer     = 0x20:0xc05786aa
stack pointer           = 0x28:0xc8798a9c
frame pointer           = 0x28:0xc8798aac
code segment            = base 0x0, limit 0xfffff, type 0x1b
                       = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 11 (swi1: net)
trap number             = 12
panic: page fault

This happens if all of the following conditions are true:

- FAST_IPSEC is used (which is a prerequisite for enc(4) anyway)

- at least one IPsec SPD and SAD entry is present

- ipfw and dummynet are loaded

- there is an ipfw rule that matches incoming decapsulated IPsec packets (not ESP or IPIP, but the actual payload) on the interface where ESP packets come in

- interface enc0 is up

Then, as soon as one ESP packet is received, the kernel panics.

Kernel debugging showed that the problem lies in the three different places where ipsec_filter() (from sys/net/if_enc.c) is called: in sys/netipsec/{ipsec_input.c,ipsec_output.c,xform_ipip.c}. When ipsec_filter() passes a packet (via pfil hooks) to ipfw, and it gets consumed by dummynet, ipsec_filter() returns 0 and the mbuf pointer that was passed in is set to NULL. However, the ipsec_filter() callers do not check for this and continue processing with the NULL mbuf.

The fix is to add appropriate checks to stop processing if the mbuf pointer was set to NULL.

Furthermore, it seems that ipfw doesn't match inbound packets on enc0, but on the interface where the corresponding ESP packet came in instead; this is not surprising though, as ipfw_check_in() in sys/netinet/ip_fw_pfil.c ignores its "ifp" argument, and if_enc doesn't replace the ifp in the mbuf with the one for enc0. It works for outbound packets on enc0 as ipfw_check_out() honors the ifp argument.
>How-To-Repeat:
On a FreeBSD 6.2 machine, compile a kernel with

options FAST_IPSEC
device enc
device crypto

Set up an IPsec tunnel to a remote system (manually with setkey or using ipsec-tools/racoon); assume that the local endpoint is on interface em0. Load ipfw and dummynet, and then add rules as follows:

ipfw pipe 100 config bw 10Mbps
ipfw add 100 pipe 100 ip from any to any in via em0
ipfw add 60000 pass ip from any to any

Configure enc0 up:

ifconfig enc0 up

Send a single packet through the IPsec tunnel from the remote system (or just ping the remote system from the machine under test), and you'll get a kernel panic immediately.
>Fix:
See the attached patch for the kernel panic issue (but not the general inconsistency of ipfw not matching inbound packets on enc0).

Patch attached with submission follows:

--- sys/netipsec/ipsec_input.c.orig	Tue Jul 25 01:20:59 2006
+++ sys/netipsec/ipsec_input.c	Sun Dec 23 16:20:07 2007
@@ -450,9 +450,12 @@
 	 */
 	ipsec_bpf(m, sav, AF_INET);
 
-	if (prot != IPPROTO_IPIP)
+	if (prot != IPPROTO_IPIP) {
 		if ((error = ipsec_filter(&m, 1)) != 0)
 			return (error);
+		if (m == NULL)		/* consumed by filter */
+			return 0;
+	}
 #endif
 
 	/*
--- sys/netipsec/ipsec_output.c.orig	Tue Jul 25 01:20:59 2006
+++ sys/netipsec/ipsec_output.c	Sun Dec 23 16:20:13 2007
@@ -364,6 +364,8 @@
 	/* pass the mbuf to enc0 for packet filtering */
 	if ((error = ipsec_filter(&m, 2)) != 0)
 		goto bad;
+	if (m == NULL)		/* consumed by filter */
+		goto bad;
 #endif
 
 	if (!tunalready) {
--- sys/netipsec/xform_ipip.c.orig	Sat Dec  9 14:05:49 2006
+++ sys/netipsec/xform_ipip.c	Sun Dec 23 16:20:22 2007
@@ -350,6 +350,8 @@
 	/* pass the mbuf to enc0 for packet filtering */
 	if (ipsec_filter(&m, 1) != 0)
 		return;
+	if (m == NULL)		/* consumed by filter */
+		return;
 #endif
 
 	/*


>Release-Note:
>Audit-Trail:
>Unformatted: