jail(8) bug with vnet & non-vnet jails running at same time?

[Dan Langille]

> On Aug 2, 2020, at 2:49 PM, Ernie Luzar <luzar722 at gmail.com> wrote:
> 
> Dan Langille wrote:
>>> On Aug 2, 2020, at 1:48 PM, Ernie Luzar <luzar722 at gmail.com> wrote:
>>> 
>>> Hello list;
>>> Please review configuration looking for something I may have missed. Hopping someone can suggest something that will change the behavior eliminating the problem.
>>> 
>>> 
>>> Equipment. Real hardware, 12.1 release, amd64 dual cpu.
>>> 
>>> Description;
>>> non-vnet jails and vnet jails using the bridge/epair method can ping the public internet when only non-vnet jails are started at a time or when only vnet jails are started at a time. But when both non-vnet jails and vnet jails are started together then neither one can ping the public internet. The order of the jails definitions in the jail.conf file has no effect on changing what is happening.
>>> 
>>> Bug description:
>>> When non-vnet jails are started their ip addresses are added to the NIC facing the public AFTER the public ip address and the non-vnet jail has access to the public internet. But when both non-vnet jails and vnet jails are started at the same time then the non-vnet jails ip addresses gets added before the public ip address of the NIC facing the public internet causing the host to lose all access to the public internet. This seems to be a jail(8) bug.
>>> 
>>> It makes no difference which command method is used to start and stop the jails.
>>> Service jail onestart jailname   or  jail –cv jailname
>> This may be related to my twitter rant about vnet problems in my own jails:
>>  https://twitter.com/DLangille/status/1289944047763693569
>> The symptoms you describe to similar to my own.  I cannot access ports on jails on the same host, but I can access ports on other hosts.
> 
> Your twitter posts are all pf firewall related.  From what I can tell you are using local only vnet jails and want to talk between them.
> 
> Do you have any non-vnet jails running on the host where the 2 vnet jails are running?
> 
> Do you have any local only vnet jails working on any other systems?

One of those two jails in question is vnet, the other is not.  There are many non-vnet jails on this host, only one vnet.

> To me knowledge there is only 1 way to have local only vnet jails to talk to each other.  Do not assign ip address to epairXa or to the bridge. Only assign an ip address to epairXb the interface in the vnet jail. All the vnet jails you want to be local only have to be members on the same bridge.

I will look at that for this jail.  Thank you.


-- 
Dan Langille - BSDCan / PGCon
dan at langille.org