multiple interfaces for jail.conf(1) and jail_set(2)

[Isaac (.ike) Levy]

Thanks Ernie,

But, that straight out did not work for me,

> On Dec 13, 2016, at 9:09 PM, Ernie Luzar <luzar722 at gmail.com> wrote:
> 
> Isaac (.ike) Levy wrote:
>> Hi All,
>> Can I specify multiple IP interfaces and assign IP’s to them using jail.conf?
>> I have jails with IPv4/IPv6 addresses on multiple physical interfaces, as well as assigning a loopback.
>> I have not found answers in the respective man pages or digging online.
>> I’m finally starting to poke around to start using the impressively simple jail.conf subsystem to manage jails.  I have been managing jails with simple custom start scripts since 99’, and custom devfs rulesets since ~2006, so jail.conf(1) and jail_set(2) are a big welcome change for me- really awesome and clean :)
>> --
>> Additional detail to clarify my loopback use:
>> In general, I always assign each jail it’s own a loopback IP somewhere in the RFC5735 specified range, 127.0.0.0/8 - (simply saving 127.0.0.1 for the jailing host), and then I simply set localhost to point at it’s IP in /etc/hosts for the jail.  On the host, I simply add the IP alias to lo0 like any other interface.
>> This is often overlooked in common jailing practice, but often eliminates complexity and confusion for many userland daemons.  For full Virtual Server applications, loopback is simply dotting the i’s and crossing the t’s.
>> I can see how localhost would be challenging to automate for easy jail.conf configuration, mostly, in picking a loopback IP for the jail and not letting that get messy- etc…
>> Thanks in advance for any info!
>> Best,
>> .ike
> 
> Using native jail.conf you can assign multiple NICs with both ipv4 & ipv6 ip address. By native I mean use the jail(8) command to start/stop your jails IE. not [service jail start] command. Use this format
> ip.addr = "rlo:10.0.10.02,xl0:10.20.10.07,lo0:127.10.0.02" This will also automatically create and remove the required aliases.

That does not appear to work- which is sad, I was excited by the syntax.

I am getting the following error,

r# jail -c myjail
jail: medial: ip4.addr: not an IPv4 address: em0:10.0.0.22
jail: myjail: ip6.addr: not an IPv6 address: em0:2:2:2:2::22
# uname -r
11.0-RELEASE-p2

My jail.conf contains precisely the following,

myjail {
    path = /foo/bar;
    mount.devfs;
    host.hostname = bar.blackskyresearch.net;
    ip4.addr = "em0:10.0.0.22,lo0:127.0.0.22";
    ip6.addr = "em0:2:2:2:2::22";
    exec.start = "/bin/sh /etc/rc";
    exec.stop = "/bin/sh /etc/rc.shutdown";
}

Noteworthy- the error thrown for ip4.addr does not even get to mentioning the second listed IP on lo0.

> 
> A word about loopback. Just like on the host, most services do not use the loopback interface, this is also true for jailed services. Only services that default to using the loopback interface need one defined in the jail to work correctly.

Sure sometimes, but not always.  While 127.0.0.1 is hardcoded into many apps and configs, this is certainly more controllable in my world where I can physically slap whomever wrote the daemon with hardcoded IP’s- even for using local inet sockets :)

> 
> Take note, the services that use the loopback interface default to using 127.0.0.1 ip address. For a service in a jail that uses loopback MUST have it's configuration changed to use the 127.10.0.02 ip address assigned on the jails jail.conf ip.addr parameter. No service in a jail can be assigned the hosts 127.0.0.1 ip address.

Certainly.  Yet, I’ve found very few headaches after changing a /etc/hosts to reflect the localhost IP for the jail.  “localhost” just resolves, as it should.

> 
> I recommend you check out these ports,
> jail-primer gives background on jails across Freebsd releases.

I believe I gave the author of that document extensive feedback when it was originally authored- as a submission rewrite for the handbook.

While this jail-primer doc was filled with many useful and practical words of advice, it was also a document which I provided a great deal of constructive feedback for the author, (pre 9.2 release).

I was particularly worried about the way the “jail cell” vocabulary abstraction was introduced and used.  I cited a relentless “use my port” approach to jail administration.  And finally, in that doc, there was far too much of an overall fundamental shift away from base UNIX ways of doing things- and even the FreeBSD way of doing things.  I find documentation like this to be frustrating for oldschoolers because it is not concise or technically informative, and detracts for new users- by presenting jail(8) in a manner which is abstracted into something so from the FreeBSD operating system.

On a quick skim, the jail-primer project you posted appears to be roughly the same document- and it also does not have the information about IP interfaces jail.conf syntax you mention above.

> qjail a utility that simplifies jail admin.

Thanks, but I’m not really interested in qjail or else I would have asked about it wherever they run their list!

While I do see tools like qjail, good ol’ ezjail, and iocage as being very valuable, they have little to do with my question.

--
Back to the original post- did I do something wrong or interpret your instructions incorrectly?
Thanks!

Best,
.ike



> 
> 
> 
> 
> 
> 
> 
> 
>