Re: bastille : poudriere not working in jail: jail: jail:_set: Operation not permitted!

On Mon, 28 Feb 2022 17:11:27 +0100
Michael Gmelin <grembo@freebsd.org> wrote:

[...]
schnipp
[...]
> > 
> > poudriere jail -l:
> > 
> > # poudriere jail -l
> > JAILNAME VERSION ARCH METHOD TIMESTAMP PATH
> > 123-amd64 12.3-RELEASE amd64
> > url=https://download.freebsd.org/releases/a ... 3-RELEASE/ 2022-02-24
> > 14:14:25 /mnt/poudriere/jails/123-amd64 130-amd64 13.0-RELEASE amd64
> > url=https://download.freebsd.org/releases/a ... 0-RELEASE/ 2022-02-24
> > 14:11:32 /mnt/poudriere/jails/130-amd64
> > 
> > The jail.conf for this specific jail is as follows:
> > 
> > [...]
> > pulverfass-001 {
> > devfs_ruleset = 13;
> > enforce_statfs = 1;
> > exec.clean;
> > exec.consolelog =
> > /mnt/extensions/bastille/logs/pulverfass-001_console.log; exec.start
> > = '/bin/sh /etc/rc'; exec.stop = '/bin/sh /etc/rc.shutdown';
> > host.hostname = XXXXXXXXX;
> > mount.devfs;
> > mount.fstab = /mnt/extensions/bastille/jails/pulverfass-001/fstab;
> > path = /mnt/extensions/bastille/jails/pulverfass-001/root;
> > securelevel = 0;
> > 
> > vnet;
> > vnet.interface = e0b_bastille4;
> > exec.prestart += "jib addm bastille4 igb0";
> > exec.prestart += "ifconfig e0a_bastille4 description \"vnet host
> > interface for Bastille jail pulverfass-001\""; exec.poststop += "jib
> > destroy bastille4";
> > 
> > allow.mount;
> > allow.mount.fdescfs;
> > allow.mount.devfs;
> > allow.mount.tmpfs;
> > allow.mount.nullfs;
> > allow.mount.procfs;
> > allow.mount.linsysfs;
> > allow.mount.linprocfs;
> > allow.mount.zfs;
> > 
> > allow.chflags;
> > allow.raw_sockets;
> > allow.socket_af;
> > allow.sysvipc;
> > 
> > linux = new;
> > 
> > exec.created += "/sbin/zfs jail ${name} BUNKER00/poudriere";
> > exec.start += "/sbin/zfs mount -a";
> > exec.poststop += "/sbin/zfs unjail BUNKER00/poudriere";
> > 
> > }
> > [...]
> > 
> > Tracking the execution of the build process by issuing
> > 
> > poudriere -x bulk ...
> > 
> > and examin the resulting trace doesn' tgive me any hint, the error
> > reported above immediately occurs when the jail is about to be
> > started:
> > 
> > + set -u +x
> > + jail -c persist 'name=123-amd64-head-default'
> > 'path=/mnt/poudriere/data/.m/ \ 123-amd64-head-default/ref'
> > 'host.hostname=basehost.local.domain' \ 'ip4.addr=127.0.0.1'
> > 'ip6.addr=::1' allow.chflags allow.sysvipc jail: jail_set: Operation
> > not permitted
> > + exit_handler
> > [...]
> > 
> > Searching the net revealed some issues with setting IP4 and IP6 in
> > poudriere, but those findings are dated back to 2017 and 2014 and I
> > guess this is solved right now.
> > 
> > The difference between our manually jail.conf driven setup and the
> > XigmaNAS/bastille based one is, bastille uses jib/netgraph based
> > seutups of the vnet and the ip4/ip6 is setup from rc.conf, while we
> > use epair in the other world and the ip is setup from withing the
> > jail definition in jail.conf.
> > 
> > I'm out of ideas here and after two days of trial and error and
> > trying to understand what's going on lost ... Any hints or tipps?
> > 
> > Thanks in advance,
> > 
> > O. Hartmann  
> 
> Hi Oliver,
> 
> I don't see `children.max` set in any of the configuration you shared
> above.
> 
> Cheers
> Michael
> 

Hello Michael,

bummer! I was so selfconfident because I copied the initial config from a working test
and had this attribute already set that I never checked again its existence - and started
reorganizing the jail.conf attributes ... 
A fine observation and a full hit: after setting children.max= 128; the poudriere jail
started working ... didn't wait for the finish so far.

I'm sorry for the noise - thanks for you eyes ...

Kind regards,

Oliver