I'm getting sick - Problems filtering IPv6.

Wed Aug 2 16:02:56 UTC 2006

[please do not cut the audit trail from your replys - it really helps to have 
all information in one email]

Short recap for everybody:  Using pf stateful rules for inet6 fails for 
connections originating from the firewall itself to a service running on the 
same box.  Culprit seems to be interface selection in inet6 (switching 
between the interface that has the address configured and lo0).  See below.

On Wednesday 02 August 2006 16:21, Frank Steinborn wrote:
> Max Laier wrote:
> > > Hello Max,
> > >
> > > a state is created, yes:
> > >
> > > self tcp 2001:1638:17ad::3[53] <- 2001:1638:17ad::3[62810]
> > > SYN_SENT:ESTABLISHED
> > >    [342525613 + 65536](+2469478632) wscale 1  [3355548528 +
> > > 65537](+82545723) wscale 1
> > >    [1845438366 + 4880](+1776883750)  [3423429433 + 65535](+3331864375)
> > >    age 00:37:53, expires in 00:00:59, 2204:15980 pkts, 107106:2269450
> > > bytes
> > >    age 01:22:57, expires in 00:01:00, 5472:42944 pkts, 324485:6199453
> > > bytes
> > >    age 02:00:22, expires in 00:00:59, 11249:53620 pkts, 967458:7637333
> > > bytes
> > >
> > >
> > > Strange thing :-(
> >
> > Indeed, and far from what I expected to see.  These states exist for a
> > long time and have seen lots of packets in both directions.  Are you sure
> > you copied the right counters for that state?  Can you please enable
> > extended logging with "pfctl -x misc" and report any related messages
> > from console. Also, please recheck pfctl -vss for the right state
> > counters.  I do get this right, the "telnet 2001:1638:17ad::3 53" stalled
> > right away?
>
> You are correct, I probably tried to many telnets so that states are
> left. I did it again, and here is the state from the telnet:
>
> self tcp 2001:1638:17ad::3[53] <- 2001:1638:17ad::3[59655]
> SYN_SENT:ESTABLISHED
>     [2728554970 + 65536](+2360520929) wscale 1  [1947983223 +
> 65537](+3290820275) wscale 1
>     age 00:00:02, expires in 00:00:28, 1:1 pkts, 84:84 bytes, rule 45
>
> There is nothing logged on the console due to pfctl -x misc, so i
> tried pfctl -x loud. However, the only thing i see are some
>
> "fingerprinted 84.191.87.127:64944  8576:118:0:48:403 (4)
> (TS=,M=536,W=0)" (IP's vary, of course, can't find v6 however)
>
> and
>
> "osfp no match against 3400000".
>
> But i guess that's not important here.
>
> And yes, you got it right - if I "telnet 2001:1638:17ad::3 53" it just
> stalls and times out after some time (even when i try block-policy
> return). But only on the box itself where pf and named is running,
> other boxes can access it fine.

Using this simple test ruleset, I was able to spot the problem:

pass quick on lo0 all
pass quick on bge0 inet all
block drop log all
pass in log-all on bge0 inet6 proto tcp from any to 3000::1 port = ssh \
  flags S/SA keep state

tcpdump on pflog0 shows that the initial SYN is coming from bge0.  The reply 
then comes via lo0 and matches the state (if state-policy is floating).  The 
third packet (again via bge0) then does no longer match the state - however:

17:51:17.594100 rule 3/0(match): pass in on bge0: 3000::1.54335 > 3000::1.22: 
S 3551126931:3551126931(0) win 65535 <mss 1440,nop,wscale 1,nop,nop,timestamp 
2188256 0,sackOK,eol>

17:51:17.594150 rule 3/0(match): pass out on lo0: 3000::1.22 > 3000::1.54335: 
S 3700289867:3700289867(0) ack 3551126932 win 65535 <mss 1440,nop,wscale 
1,nop,nop,timestamp 2188256 2188256,sackOK,eol>

17:51:17.594157 rule 2/0(match): block in on bge0: 3000::1.22 > 3000::1.54335: 
S 3700289867:3700289867(0) ack 3551126932 win 65535 <mss 1440,nop,wscale 
1,nop,nop,timestamp 2188256 2188256,sackOK,eol>

Can somebody with a recent OpenBSD box please check the behavior of inet6 
routing/interface selection there and report?

As for a fix, I suspect that fixing the inet6 routing/interface selection will 
be far from trivial (and I have to check with the RFCs to see if we may 
change it at all).  Something is certainly broken in inet6-land as *none* of 
these packets show up in any bpf - not on lo0 and neither on bge0.

-- 
/"\  Best regards,                      | mlaier at freebsd.org
\ /  Max Laier                          | ICQ #67774661
 X   http://pf4freebsd.love2party.net/  | mlaier at EFnet
/ \  ASCII Ribbon Campaign              | Against HTML Mail and News
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-pf/attachments/20060802/1bf6e088/attachment.pgp