pf not seeing inbound packets on netgraph interface

Ed Carrel azanar at
Tue Jan 3 06:40:55 UTC 2012

Hi freebsd-questions,

I am running into a roadblock getting PF to filter traffic on a Netgraph
interface representing an L2TP/IPSec connection. I have done some narrowing
down of the problem, but was hoping to get some advice on figuring out
where to go digging next, or things to try.

Also, please let me know if I should ask this on another list.

For context, here is what I have setup so far: I am running FreeBSD 8.2
with IPSec support compiled into the kernel. Here's the details from uname:

# uname -a
FreeBSD **** 8.2-RELEASE-p4 FreeBSD 8.2-RELEASE-p4 #2: Sun Nov 27 04:12:16
PST 2011 ****  i386

I am following what seems like a typical setup of racoon(8) and setkey(8),
and am having mpd5 handle construction of the L2TP nodes in netgraph. I can
provide the details on the configuration of each of these, if desired. When
I startup racoon in the foreground and ask mpd to construct an L2TP link, I
can see both the IPSec tunnel and the L2TP link establish without a
problem. I am able to ping the remote end, and, if I set up a routing rule,
can contact and ssh to hosts at the other end of the tunnel.

Here's how netgraph sees the world when thing are established:

# ngctl list
There are 13 total nodes:
  Name: <unnamed>       Type: ksocket         ID: 000001b3   Num hooks: 1
  Name: <unnamed>       Type: l2tp            ID: 000001b1   Num hooks: 3
  Name: <unnamed>       Type: socket          ID: 000001b0   Num hooks: 1
  Name: ng0             Type: iface           ID: 000001b6   Num hooks: 1
  Name: ngctl26124      Type: socket          ID: 000001bd   Num hooks: 0
  Name: ngctl19375      Type: socket          ID: 000000ba   Num hooks: 0
  Name: mpd25875-stats  Type: socket          ID: 000001b8   Num hooks: 0
  Name: mpd25875-WPLink-lt Type: tee             ID: 000001af   Num hooks: 2
  Name: mpd25875-cso    Type: socket          ID: 000001ad   Num hooks: 0
  Name: mpd25875-eso    Type: socket          ID: 000001ae   Num hooks: 0
  Name: mpd25875-lso    Type: socket          ID: 000001ac   Num hooks: 1
  Name: mpd25875-WPBundle-1 Type: ppp             ID: 000001b7   Num hooks:
  Name: ng0-tee         Type: tee             ID: 000001b9   Num hooks: 2

The problem I have is that PF only sees traffic on the outbound side of the
netgraph interface. But, the rest of the network stack appears to see data
going both ways:

# ifconfig ng0
mtu 1322
inet --> netmask 0xffffffff

# pfctl -vvs Interfaces -i ng0
Cleared:     Sun Dec 25 21:14:44 2011
References:  [ States:  0                  Rules: 9                  ]
 In4/Pass:    [ Packets: 0                  Bytes: 0                  ]
In4/Block:   [ Packets: 0                  Bytes: 0                  ]
 Out4/Pass:   [ Packets: 5555               Bytes: 446225             ]
Out4/Block:  [ Packets: 622                Bytes: 56336              ]
 In6/Pass:    [ Packets: 0                  Bytes: 0                  ]
In6/Block:   [ Packets: 0                  Bytes: 0                  ]
 Out6/Pass:   [ Packets: 0                  Bytes: 0                  ]
Out6/Block:  [ Packets: 0                  Bytes: 0                  ]

# netstat -I ng0 -bn
Name    Mtu Network       Address              Ipkts Ierrs Idrop     Ibytes
   Opkts Oerrs     Obytes  Coll
ng0    1322 <Link#8>                              56     0     0       5069
      98     0       6032     0
ng0    1322              56     -     -       5069
      54     -       3472     -

I have stood up this interface several times, hence the differing numbers
between the PF and netstat. The cause for concern is the lack of packets
going through PF when inbound on ng0 -- no problem both seeing them and
applying rules going outbound. There isn't a peep about the inbound traffic
within pflog0, either.

I can see traffic going both ways in tcpdump, and nothing looks peculiar
about the packets.

# tcpdump -c 10 -i ng0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ng0, link-type NULL (BSD loopback), capture size 96 bytes
22:06:37.201732 IP > Flags [S], seq
3442571726, win 65535, options [mss 1282,nop,wscale 3,sackOK,TS val
694436002 ecr 0], length 0
22:06:37.263336 IP > Flags [S.], seq
1974232057, ack 3442571727, win 14480, options [mss 1282,sackOK,TS val
370681934 ecr 694436002,nop,wscale 7], length 0
22:06:37.263577 IP > Flags [.], ack 1, win
8255, options [nop,nop,TS val 694436064 ecr 370681934], length 0
22:06:37.282835 IP > Flags [P.], ack 1, win
114, options [nop,nop,TS val 370681940 ecr 694436064], length 21
22:06:37.283931 IP > Flags [P.], ack 22,
win 8255, options [nop,nop,TS val 694436084 ecr 370681940], length 40
22:06:37.300729 IP > Flags [.], ack 41, win
114, options [nop,nop,TS val 370681945 ecr 694436084], length 0
22:06:37.300943 IP > Flags [P.], ack 22,
win 8255, options [nop,nop,TS val 694436101 ecr 370681945], length 848
22:06:37.304154 IP > Flags [P.], ack 41,
win 114, options [nop,nop,TS val 370681945 ecr 694436084], length 984
22:06:37.372460 IP > Flags [.], ack 889,
win 127, options [nop,nop,TS val 370681967 ecr 694436101], length 0
22:06:37.372663 IP > Flags [P.], ack 1006,
win 8255, options [nop,nop,TS val 694436173 ecr 370681945], length 24
10 packets captured
22 packets received by filter
0 packets dropped by kernel

As I noted above, I can interact with hosts over the tunnel so long as PF
blindly passes traffic. Attempting to do any sort of stateful connection
tracking causes PF to litter /var/log/messages with notices of a "loose
state match," which I think is to be expected since it is only seeing the
outbound half the network conversation.

Ideas on things to try or investigate next? I can provide a paste of any
relevant config or log files, just let me know.


Ed Carrel

More information about the freebsd-questions mailing list