Firewalling jails and lo0

Ernie Luzar luzar722 at gmail.com
Sun Aug 7 14:20:25 UTC 2016


Niklaas Baudet von Gersdorff wrote:
> Bjoern A. Zeeb [2016-08-06 20:02 +0000] :
> 
>> I am curious about this.  Can you give me an (obfuscated) example?  (if 
>> you want in private email)
> 
> -- $ jls -v
>    JID  Hostname                      Path
>         Name                          State
>         CPUSetID
>         IP Address(es)
>   [...]
>      7  mx.box-hlm-03.niklaas.eu      /usr/local/jails/smtp1
>         smtp1                         ACTIVE
>         8     
>         10.3.8.1       
>         fd16:dcc0:f4cc:3::8:1
>   [...]
>     24  proxy1.box-hlm-03.niklaas.eu  /usr/local/jails/proxy1
>         proxy1                        ACTIVE
>         5     
>         10.3.2.1       
>         10.77.2.1      
>         fd16:dcc0:f4cc:3::2:1
>         fd16:dcc0:f4cc:77::2:1
>   [...]
> --
> 
> -- $ ifconfig lo1
>   lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
>     options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
>   [...]
>     inet 10.3.8.1 netmask 0xffff0000 
>     inet6 fd16:dcc0:f4cc:3::8:1 prefixlen 64 
>   [...]
>     inet 10.3.2.1 netmask 0xffff0000 
>     inet 10.77.2.1 netmask 0xffff0000 
>     inet6 fd16:dcc0:f4cc:3::2:1 prefixlen 64 
>     inet6 fd16:dcc0:f4cc:77::2:1 prefixlen 64 
>   [...]
>     nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
> --
> 
> The following is a /full/ output:
> 
> -- $ ifconfig lo0
>   lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
>     options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
>     inet6 ::1 prefixlen 128 
>     inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 
>     inet 127.0.0.1 netmask 0xff000000 
>     nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
> --
> 
> So, as you can see, the jails only have IP addresses on lo1 and
> none of them has one on lo0. To make that clear:
> 
> -- $ jexec smtp1 ifconfig
>   [...]
>   lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
>     options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
>     nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
>   lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
>     options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
>     inet 10.3.8.1 netmask 0xffff0000 
>     inet6 fd16:dcc0:f4cc:3::8:1 prefixlen 64 
>     nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
>   [...]
> --
> 
> In my pf.conf I have the following. This is a simplified extract:
> 
> -- /etc/pf.conf
>    1   ext_if            = vtnet0
>    2   jail_if           = lo1
>    3  
>    4   table <proxy>       persist
>    5   table <mail>        persist
>    6  
>    7   set skip on lo0 
>    8  
>    9   nat on $ext_if from { <proxy> <mail> } to any -> <me>
>   10  
>   11   block log all
>   12  
>   13   pass out all keep state
>   14  
>   15   pass in on $jail_if proto tcp from <proxy> to <mail>    port { <some-ports> }
> --
> 
> As you can see I have a principal block in line 11, and skip is
> set on lo0 solely. That said, I block on lo1. Because of this,
> I pass on lo1 in line 19. I thought this is necessary.
> 
> However, here comes the thing: Although the jails have IP addresses
> attached to lo1 only, I can see traffic like the following:
> 
> -- $ tcpdump -nettti lo0 host 10.3.2.1
>   00:00:00.023424 AF IPv4 (2), length 64: 10.3.2.1.51096 > 10.3.8.1.9025: Flags [S], seq 4205430985, win 65535, options [mss 16344,nop,wscale 6,sackOK,TS val 349909592 ecr 0], length 0
>   00:00:00.000064 AF IPv4 (2), length 64: 10.3.8.1.9025 > 10.3.2.1.51096: Flags [S.], seq 3921176095, ack 4205430986, win 65535, options [mss 16344,nop,wscale 6,sackOK,TS val 3273771227 ecr 349909592], length 0
>   00:00:00.000023 AF IPv4 (2), length 56: 10.3.2.1.51096 > 10.3.8.1.9025: Flags [.], ack 1, win 1275, options [nop,nop,TS val 349909592 ecr 3273771227], length 0
> --
> 
> -- $ tcpdump -nettti lo0 host fd16:dcc0:f4cc:3::8:1                                          
>   tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>   listening on lo0, link-type NULL (BSD loopback), capture size 65535 bytes
>   00:00:00.000000 AF IPv6 (28), length 84: fd16:dcc0:f4cc:3::2:1.35851 > fd16:dcc0:f4cc:3::8:1.25: Flags [S], seq 3339315349, win 65535, options [mss 16324,nop,wscale 6,sackOK,TS val 352469079 ecr 0], length 0
>   00:00:00.000035 AF IPv6 (28), length 84: fd16:dcc0:f4cc:3::8:1.25 > fd16:dcc0:f4cc:3::2:1.35851: Flags [S.], seq 3726000680, ack 3339315350, win 65535, options [mss 16324,nop,wscale 6,sackOK,TS val 306734766 ecr 352469079], length 0
>   00:00:00.000044 AF IPv6 (28), length 76: fd16:dcc0:f4cc:3::2:1.35851 > fd16:dcc0:f4cc:3::8:1.25: Flags [.], ack 1, win 1274, options [nop,nop,TS val 352469079 ecr 306734766], length 0
>   00:00:05.060320 AF IPv6 (28), length 107: fd16:dcc0:f4cc:3::8:1.25 > fd16:dcc0:f4cc:3::2:1.35851: Flags [P.], seq 1:32, ack 1, win 1274, options [nop,nop,TS val 306739827 ecr 352469079], length 31
>   00:00:00.000113 AF IPv6 (28), length 76: fd16:dcc0:f4cc:3::8:1.25 > fd16:dcc0:f4cc:3::2:1.35851: Flags [F.], seq 32, ack 1, win 1274, options [nop,nop,TS val 306739827 ecr 352469079], length 0                                00:00:00.000025 AF IPv6 (28), length 76: fd16:dcc0:f4cc:3::2:1.35851 > fd16:dcc0:f4cc:3::8:1.25: Flags [.], ack 33, win 1273, options [nop,nop,TS val 352474140 ecr 306739827], length 0                                        00:00:00.000413 AF IPv6 (28), length 76: fd16:dcc0:f4cc:3::2:1.35851 > fd16:dcc0:f4cc:3::8:1.25: Flags [F.], seq 1, ack 33, win 1274, options [nop,nop,TS val 352474140 ecr 306739827], length 0
> --
> 
> As you can see, this is on lo0 although the jails don't have an
> IP address on it. That said, restricting traffic on lo1 doesn't
> make any sense because the jails use lo0 anyway.
> 
>> Are these ::1 connections, link-local addresses (unlikely as they should 
>> not be visible to jails), or full IP?
> 
> As you can see, they are full IP.
> 
>> And what’s the routing table entry in the base system for them?
> 
> Have a look at the following output of netstat (I removed some lines and cells):
> 
> -- $ netstat -rn
>   Routing tables
> 
>   Internet:
>   Destination        Gateway            Flags      Netif Expire
>   default            <          >       UGS      vtnet0
>   10.0.0.0/8         link#4             U          tap0
>   10.3.2.1           link#3             UH          lo1
>   10.3.8.1           link#3             UH          lo1
>   10.77.2.1          link#3             UH          lo1
>   127.0.0.1          link#2             UH          lo0
> 
>   Internet6:
>   Destination                       Gateway                       Flags      Netif Expire
>   ::/96                             ::1                           UGRS        lo0
>   default                           <                  >          UGS      vtnet0
>   ::1                               link#2                        UH          lo0
>   ::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
>   <                       >         link#1                        U        vtnet0
>   <                               > link#1                        UHS         lo0
>   fd16:dcc0:f4cc:3::/64             link#3                        U           lo1
>   fd16:dcc0:f4cc:3::1               link#4                        UHS         lo0
>   fd16:dcc0:f4cc:3::2:1             link#3                        UHS         lo0
>   fd16:dcc0:f4cc:3::8:1             link#3                        UHS         lo0
>   fd16:dcc0:f4cc:77::/64            link#3                        U           lo1
>   fd16:dcc0:f4cc:77::2:1            link#3                        UHS         lo0
>   fe80::/10                         ::1                           UGRS        lo0
>   fe80::%vtnet0/64                  link#1                        U        vtnet0
>   fe80::<                >%vtnet0   link#1                        UHS         lo0
>   fe80::%lo0/64                     link#2                        U           lo0
>   fe80::1%lo0                       link#2                        UHS         lo0
>   fe80::%tap0/64                    link#4                        U          tap0
>   fe80::<             >%tap0        link#4                        UHS         lo0
>   ff01::%vtnet0/32                  <                      >%vtnet0 U        vtnet0
>   ff01::%lo0/32                     ::1                           U           lo0
>   ff01::%lo1/32                     fd16:dcc0:f4cc:3::1:1         U           lo1
>   ff01::%tap0/32                    fd16:dcc0:f4cc:3::1           U          tap0
>   ff02::/16                         ::1                           UGRS        lo0
>   ff02::%vtnet0/32                  <                      >%vtnet0 U        vtnet0
>   ff02::%lo0/32                     ::1                           U           lo0
>   ff02::%lo1/32                     fd16:dcc0:f4cc:3::1:1         U           lo1
>   ff02::%tap0/32                    fd16:dcc0:f4cc:3::1           U          tap0
> --
> 
>> especially, do they have any IP address assigned to lo0 in them at all?
> 
> No, they don't.
> 
>     Niklaas


I believe the loopback interface lo1 needs 127.0.0.0/8 ip address to 
enable loopback functionally, and the ip address has to be a different 
sub-net. IE 127.0.10.1 for lo1 while the hosts lo0 uses 127.0.0.1


More information about the freebsd-questions mailing list