configured ipv6 in vnet jail w. ubuntu 22.04 (jammy) not … seen by linux tools (IPV&_RECVERR: Protocol not available)

From: Mathias Picker <Mathias.Picker_at_virtual-earth.de>
Date: Sat, 12 Nov 2022 18:46:31 UTC
Hi all,

tl;dr:

can I convince/configure linux emulation in a jail to show IPv6 as 
supported? FreeBSD ifconfig in the jail has it configured,
linux ’sysctl net.ipv6.conf.all.disable_ipv6=0’ does not work.

Description:

This is on a 13.1-RELEASE-p2 system.

I have used debootstrap to create a jail with Ubuntu 22.04.

Some parts of my /etc/jail.conf:

exec.clean;
exec.start="sh /etc/rc";
exec.stop="sh /etc/rc.shutdown";
exec.prestart="logger starting jail $name ...";
exec.poststart="logger jail $name has started";
exec.prestop="logger shutting down jail $name";
exec.poststop="logger jail $name has shut down";

# generic hostnames
host.hostname="$name.goodhope.local";

# vnet jails
vnet;
vnet.interface="${name}_j";
exec.prestart+="/usr/local/bin/jailtobridge $name jailbridge0";
exec.poststop+="/sbin/ifconfig jailbridge0 deletem 
${name}_b;/sbin/ifconfig ${name}_b destroy";

exec.consolelog="/var/log/jails/$name-console.log";

litreview {
mount.fstab="/jails/fstabs/fstab.litreview";
allow.mount;
allow.raw_sockets;
allow.read_msgbuf;
allow.socket_af;
sysvmsg;
sysvsem;
sysvshm;
mount.devfs;
exec.start = "/bin/dash /etc/rc3.d/S01networking-fbsd";
persist;
}


I then copied/linked the freebsd tools ifconfig, sysctl and route 
from /rescue into this to configure networking.

In /etc/jail.conf I just startet a small script to call ifconfig, 
route and sysctl (to switch off ipfw) and used ’persist;’ to keep 
the jail running.

- - - - -  networkinit-fbsd - - - - - - - - 
#!/bin/sh -e
PATH="/sbin:/bin"

. /lib/lsb/init-functions

log_daemon_msg "Starting FreeBSD network configuration"
# deactivate ipfw
/bin/sysctl net.inet.ip.fw.enable=0

# set network address & route
/bin/ifconfig litreview_j inet xxx.xxx.xxx.xxx/28
/bin/route add default xxx.xxx.xxx.xxx
/bin/ifconfig litreview_j inet6 xxxx:xxxx:…./64
/bin/route -6 add default fe80::1%litreview_j

/bin/ifconfig lo0 inet 127.0.0.1
- - - - - - - -

Result: an ubuntu 22.04 system with working IPv4 connectivity:

- - - - - - - - 
root@litreview:/home/literatur_review# uname -a
Linux litreview.goodhope.local 3.17.0 FreeBSD 13.1-RELEASE-p2 
GENERIC x86_64 x86_64 x86_64 GNU/Linux
root@litreview:/home/literatur_review# cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS"
root@litreview:/home/literatur_review# ping -4 google.de
ping: WARNING: setsockopt(ICMP_FILTER): Protocol not available
PING  (172.217.18.3) 56(84) bytes of data.
64 bytes from fra02s19-in-f3.1e100.net (172.217.18.3): icmp_seq=1 
ttl=59 time=5.05 ms
^C
---  ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 5.051/5.051/5.051/0.000 ms
- - - - - - - - 

Sadly, IPv6 does not work, which I intended to use for 
accesibility from outside (this is on a server):

- - - - - - - - 
root@litreview:/home/literatur_review# ping -6 google.de
ping: IPV6_RECVERR: Protocol not available
- - - - - - - - 

I’m not sure what to make of FreeBSDs ping output:

- - - - - - - - 
root@litreview:/home/literatur_review# ./ping google.de
PING6(56=40+8+8 bytes) 2a01:4f8:10b:3de:1:1:0:21 --> 
2a00:1450:4001:829::2003
ping: sendmsg: Permission denied
ping6: wrote google.de 16 chars, ret=-1
ping: sendmsg: Permission denied
ping6: wrote google.de 16 chars, ret=-1
^C
--- google.de ping6 statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
- - - - - - - - 

ipfw is deactiviated in the jail and does not block icmp from 
outside the jail (it works from other vnet jails just fine).

FreeBSD ifconfig sees both IPv4 and IPv6:

- - - - - - - - 
root@litreview:/home/literatur_review# /bin/ifconfig 
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
litreview_j: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> 
metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:f2:90:e2:1d:0b
        inet xxx.xxx.xxx.xxx netmask 0xfffffff0 broadcast 
        xxx.xxx.xxx.xxx
        inet6 xxxx:xxxx:xxxx:xxxx:…. prefixlen 64
        inet6 fe80::f2:90ff:fee2:1d0b%litreview_j prefixlen 64 
        scopeid 0x2
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
 - - - - - - - - 


After installing nettools I can see that linux ifconfig also only 
sees the configured IPv4 address and no IPv6. BTW:  ip sees 
nothing :(

- - - - - - - - 
root@litreview:/home/literatur_review# /usr/sbin/ifconfig 
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet xxx.xxx.xxx.xxx  netmask 255.255.255.240  broadcast 
        xxx.xxx.xxx.xxx
        ether 02:f2:90:e2:1d:0b  (Ethernet)
        RX packets 203986  bytes 277350122 (277.3 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 146633  bytes 9637488 (9.6 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo0: flags=4169<UP,LOOPBACK,RUNNING,MULTICAST>  mtu 16384
        inet 127.0.0.1  netmask [NONE SET]
        loop  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
root@litreview:/home/literatur_review# /usr/sbin/ip -4
Cannot open netlink socket: Address family not supported by 
protocol
root@litreview:/home/literatur_review# /usr/sbin/ip -6
Cannot open netlink socket: Address family not supported by 
protocol
- - - - - - - - 

The usual way to do this fails because of the missing 
/proc/sys/net/… in linprocfs:

r- - - - - - - - 
oot@litreview:/home/literatur_review# sysctl 
net.ipv6.conf.all.disable_ipv6=0
sysctl: cannot stat /proc/sys/net/ipv6/conf/all/disable_ipv6: No 
such file or directory
- - - - - - - - 

Any idea how I can convince the tools that IPv6 is available is 
very welcome!

Cheers, Mathias

-- 
Mathias Picker                        
Geschäftsführer
Mathias.Picker@virtual-earth.de

virtual earth Gesellschaft für Wissens re/prä sentation mbH
http://www.virtual-earth.de/           HRB126870
support@virtual-earth.de               Westendstr. 142
089 / 1250 3943