32-bit jail on 64-bit host

[Ian Lepore]

On Thu, 2021-04-22 at 12:32 +0200, Daniel Dettlaff via freebsd-hackers
wrote:
> If you need to run 32bit software with 64bit base system just try
> creating 64bit jail with lib32 subsystem present. Then 32bit software
> should be able to run properly in such jail, but you can't run 32bit
> jail on 64bit base as Eugene said.
> 

That is not what Eugene said, and you CAN run a 32-bit jail on a 64-bit 
host; I do so on this machine.  As Eugene said, you simply need to copy
a few selected 64-bit binaries into the jail, replacing the 32-bit
version of those programs.  That is, install the jail from a 32-bit
build or packages, and then just copy the necessary few binaries from
your host root filesystem into the jail.

It would be nice if there was a list somewhere of which binaries need
to be replaced.  I just did it by trial and error... when I ran into
things that didn't work, I tried using a 64-bit copy of that program
and if it worked: problem solved.

-- Ian

> > On 22 Apr 2021, at 10:36, Peter Blok <pblok at bsd4all.org> wrote:
> > 
> > Hi,
> > 
> > I have created a 32-bit jail on a 64-bit running 12-STABLE. The
> > jail is also build using the same source.
> > 
> > The jail gives me a 32-bit environment. I’m getting an IP address
> > and I can ping others on the same network segment.
> > 
> > But I can’t set a default route.
> > 
> > route add default 192.168.1.1
> > route: writing to routing socket: Invalid argument
> > add net default: gateway 192.168.1.1 fib 0: Invalid argument
> > 
> > # netstat -rn
> > Routing tables
> > (0) (0) UH 
> > (0) (0) U 
> > (0) (0) UHS 
> > (0) (0) UH 
> > (0) (0) U 
> > (0) (0) UHS
> > 
> > # ifconfig -a
> > lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
> > 	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
> > 	inet6 ::1 prefixlen 128
> > 	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
> > 	inet 127.0.0.1 netmask 0xff000000
> > 	groups: lo
> > 	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
> > e0b_websip: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST>
> > metric 0 mtu 1500
> > 	options=8<VLAN_MTU>
> > 	ether 0e:88:d7:20:99:80
> > 	hwaddr 02:80:ad:6e:79:0b
> > 	inet 192.168.1.205 netmask 0xffffff00 broadcast 192.168.1.255
> > 	groups: epair
> > 	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
> > 	status: active
> > 	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> > 
> > Any idea how to fix this?
> > 
> > I’m using vnet bridge
> > 
> > Peter
> >