Regression with jails/IPv6/pf

[Matthew Seaman]

So, I tried to do a routine update to the latest stable/9 yesterday
(r238771), and I found that access to the jail on my server had stopped
working.  Everything else seemed to be fine, and reverting to the
previous system (r237456 from 2012-06-22 (Boot Environments FTW)) bought
it all back to life.

After spending most of today bisecting versions and compiling kernels,
I found:

    r238177 worked absolutely fine

    r238236 accessing the jail worked, but everything was slow, as if
            DNS queries were timing out.

    r238246 lots of network timeouts everywhere: accessing the jail
            failed, but then so did accessing the main host.  So much
            so that svn couldn't update properly.

    r238256 worked fine for accessing the main host, but failed when
            trying to access the jail.

Looks like this seems to have been introduced in a batch of commits
MFC'd by bz@ (CC'd) around then.

Now, this jail is set up in an unusual way, which is why I guess I'm the
first person to be affected.  For starters, it only has IPv6
connectivity, and secondly, because I'm running some daemons there I
don't want listening on an external network socket, it's bound to the
loopback and I use firewall redirection to send traffic to it.

The jail config in /etc/rc.conf looks like this:

jail_interface="lo1"
jail_devfs_enable="YES"
jail_devfs_ruleset="devfsrules_jail_zfs"
jail_fdescfs_enable="YES"
jail_procfs_enable="YES"
jail_set_hostname_allow="NO"
jail_socket_unixiproute_only="YES"
jail_sysvipc_allow="NO"
jail_parallel_start="NO"

jail_xenophobe_hostname="xenophobe.infracaninophile.co.uk"
jail_xenophobe_rootdir="/jail/xenophobe"
jail_xenophobe_ip="fd87:cd50:2103:1:54f9:9484:e8b0:12d1"
jail_xenophobe_mount_enable="YES"
jail_xenophobe_zfs="zroot/jail/xenophobe zroot/jail/xenophobe/TimeMachine"
jail_xenophobe_params="enforce_statfs=1"

I've cloned a second loopback I/F and given the jail an address from the
IPv6 private address range (RFC4193).  Cloning the interface
isn't absolutely necessary -- exactly the same symptoms occur if I use
an alias address on lo0 -- but it makes it easier to see only jail
traffic when using tcpdump.

Then I've enabled access via the network using nat+rdr in PF, like so:

table <localnets> const { 2001:8b0:151:1::/64, \
                          81.187.76.160/29,    \
                          fd87:cd50:2103:1::/64 }

xenophobe_int="fd87:cd50:2103:1:54f9:9484:e8b0:12d1"
xenophobe_ext="2001:8b0:151:1:54f9:9484:e8b0:12d1"

[...]

nat on $ext_if_plus from $xenophobe_int to any -> $xenophobe_ext
rdr inet6 proto tcp from <localnets> to $xenophobe_ext \
     port { 22, 80, 443, 548, 4700 } -> $xenophobe_int

When trying to ssh into the jail with a kernel exhibiting this problem,
tcpdump showed that traffic was reaching the sshd in the jail and
responses were being generated, but they didn't make it out onto the net.

	Cheers,

	Matthew

-- 
Dr Matthew J Seaman MA, D.Phil.                   7 Priory Courtyard
                                                  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey     Ramsgate
JID: matthew at infracaninophile.co.uk               Kent, CT11 9PW

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 267 bytes
Desc: OpenPGP digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20120726/47cc3e9a/signature.pgp