Fighting with vnet / jails epair and so on

[Denny Schierz]

hi,

after most parts works with my bridge setups works, I want to get vnet for my jails working. In the morning I started a jail and got only the local interface back, but no epair0b. Now I did something so that I can see _all_ interfaces from outside (bridge0 / bge* / epair0* ... ) but without any IPs.
However, I'm not able to give epair0b inside the jail an ip address. I get "permission denied".

Also  it looks a bit strange:

===============
host# jexec 2 ifconfig

bge0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=80099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE>
	ether CHANGED
ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
bge1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE>
	ether CHANGED
ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported
	media: Ethernet autoselect (none)
	status: no carrier
bge2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE>
	ether CHANGED
ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported
	media: Ethernet autoselect (none)
	status: no carrier
bge3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE>
	ether CHANGED
ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
pflog0: flags=0<> metric 0 mtu 33152
ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=3<RXCSUM,TXCSUM>
ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	ether CHANGED
ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported
	id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
	maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200
	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
	member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 12 priority 128 path cost 2000
	member: bge0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 4 priority 128 path cost 55
epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether CHANGED
ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
epair0b: flags=8842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=8<VLAN_MTU>
	ether CHANGED
ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported
	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
	status: active
=======================================

# host: 
jexec 2 ifconfig epair0b 192.168.1.2 netmask 255.255.255.0 up
ifconfig: up: permission denied



# sysctl:

security.jail.enforce_statfs: 2
security.jail.mount_allowed: 0
security.jail.chflags_allowed: 0
security.jail.allow_raw_sockets: 1
security.jail.sysvipc_allowed: 1
security.jail.socket_unixiproute_only: 1
security.jail.set_hostname_allowed: 1
security.jail.jail_max_af_ips: 255
security.jail.jailed: 0

/etc/rc.conf:
=============================
jail_enable="YES"
jail_v2_enable="YES"
jail_list=""
jail_sysvipc_allow="YES"


#JAIL template
jail_list="$jail_list template"
jail_template_name="template"
jail_template_hostname="template.CHANGED"
jail_template_devfs_enable="YES"
jail_template_rootdir="/jails/template"
jail_template_mount_enable="YES"
jail_template_fstab="/etc/jails/fstabs/template"
jail_template_vnet_enable="YES"
jail_template_devfs_ruleset="devfsrules_jail"
 
#network
jail_template_exec_prestart0="ifconfig epair0 create"
jail_template_exec_prestart1="ifconfig bridge0 addm epair0a"
jail_template_exec_prestart2="ifconfig epair0a up"
jail_template_exec_earlypoststart0="ifconfig epair0b vnet template"
jail_template_exec_afterstart0="ifconfig lo0 127.0.0.1"
jail_template_exec_afterstart1="ifconfig epair0b 192.168.1.2 netmask 255.255.255.0 up"
jail_template_exec_afterstart2="route add default 130.83.160.62"
jail_template_exec_afterstart3="/bin/sh /etc/rc"
jail_template_exec_poststop0="ifconfig bridge0 deletem epair0a"
jail_template_exec_poststop1="ifconfig epair0a destroy"

===========================

Starting jail:

#/etc/rc.d/jail onestart

Configuring jails:.
Starting jails:epair0a
ifconfig: up: permission denied
route: writing to routing socket: Operation not permitted
Setting hostname: example.mydomain.com.

uname -a:

9.0-STABLE FreeBSD 9.0-STABLE #0: Tue Jan 17 09:05:42 CET 2012 

Also, some people say, I have to patch /etc/rc.d/jail (freeBSD 9-rc2) to get know the new "vnet2", other say, I don't need ... so ....

Can anybody bring some light into the darkness of jails and vnet + rc?

cu denny