32-bit jail on 64-bit host

[Dewayne Geraghty]

On 23/04/2021 1:12 am, Chris wrote:
> I do it. I don't think vnet or a bridge is necessary or perhaps even
> desirable
> in this situation. All my 32/64bit jails access the hosts net out of
> localhost
> (127.0.0.2-N) and I use pf(4) to redirect the packets.
> I have a static block of internet facing addresses. So change yours
> accordingly
> pf.conf(5)
> EXT_ADDR="W.X.Y.Z"
> ...
> set skip on { lo0, lo1 }
> ...
> nat pass on re0 from { lo1 } to any -> $EXT_ADDR
> rdr pass on re0 proto tcp from any to { lo1 } -> $EXT_ADDR
> ...
> block in
> pass out
> ...
> I add an entry in the hosts hosts(5) file, and in the jails hosts(5) for
> accounting purposes. The jails resolve.conf(5) file looks like this
> nameserver 127.0.0.1
> nameserver 127.0.0.2
> options timeout:1 attempts:1 rotate
> 
> And all gets it done for me.
> 
> HTH
> 
> --Chris
>>
>> Peter

Peter, I use a similar setup to Chris, though with ipfw. ;)

Jails have a few subtleties.  They inherit much of the network of the
base.  So you only need to think about the IP's assigned to the jail and
their assignment order.  However one particular gotcha

The jail will use the first IP address that's set in jail.conf
effectively becoming your default route for the jail.  And I recall that
localhost will also latch onto that IP address, so if its internet
facing, you'll need to think about the implications.

I'm a little paranoid so I use:
-  /etc/hosts to define localhost to be something other than the
default.  Some applications/ports behave properly IF they use localhost
for their unix sockets, rather than 127.0.0.1. (ie test what you need
and become good friends with tcpdump)
-  consider carefully your firewall rules not just internet facing but
also over lo0 :)

And to reiterate what many have said, running i386 and amd64 on an amd64
platform is fun, as there are less machines to maintain when you need,
as in our use-case, to test the operation of software for 32bit targets.
(Though we just perform a buildworld  with  TARGET_ARCH=i386
CPU_TYPE=PRESCOTT with the appropriate destination.)

I don't think your setup requires the complexity or additional
processing from bridging or vnets.