From nobody Mon Feb 28 16:58:35 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C004119E529F; Mon, 28 Feb 2022 16:58:43 +0000 (UTC) (envelope-from freebsd@walstatt-de.de) Received: from smtp6.goneo.de (smtp6.goneo.de [85.220.129.31]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4K6mmf3Z24z3NvN; Mon, 28 Feb 2022 16:58:42 +0000 (UTC) (envelope-from freebsd@walstatt-de.de) Received: from hub2.goneo.de (hub2.goneo.de [85.220.129.53]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by smtp6.goneo.de (Postfix) with ESMTPS id BF70610A32F1; Mon, 28 Feb 2022 17:58:40 +0100 (CET) Received: from hub2.goneo.de (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by hub2.goneo.de (Postfix) with ESMTPS id B364610A1E89; Mon, 28 Feb 2022 17:58:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=walstatt-de.de; s=DKIM001; t=1646067516; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9tGdWIFMYoaxMOyUXHGum8i8wzVvIMZiNUZ71xDmbTw=; b=j6i2gRLI3CQeQ+4veRo1qOkSNuh8k9/TXw3ELEgUBuScVzfDgpJ41VOECL77tmGJEdIAf7 yLT43nLkxuzD0r5DDrvC06FNxb/NwfFdkAQzaHWvexmoNnQpgFM2PZeLtPnO0DVsLaMG6Y ePHoR9HzAzj3nrL+J2PKrkc2HF1Gmy6ozvrGQ0KOJCejWUL+FwC+GpaN9MyaSv6Kx4EutA b3W0GOtj5RN/Y3UESn0amM+1WMuwR6P5K3TapjSRmgftkTZ6RLKOvdov8ELDRRxpCv82X4 i7Fa5CreCowDcPMmwYHFCZ2wfx0j9pxozeQP9PwgNPkH66yX7XAK7SwXWxTGMA== Received: from hermann (dynamic-2a01-0c22-ad8a-3000-954e-91b4-d2cb-c2d0.c22.pool.telefonica.de [IPv6:2a01:c22:ad8a:3000:954e:91b4:d2cb:c2d0]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by hub2.goneo.de (Postfix) with ESMTPSA id 6F1E610A1E85; Mon, 28 Feb 2022 17:58:36 +0100 (CET) Date: Mon, 28 Feb 2022 17:58:35 +0100 From: FreeBSD User To: Michael Gmelin Cc: FreeBSD virtualization , FreeBSD CURRENT , Mateusz Guzik Subject: Re: bastille : poudriere not working in jail: jail: jail:_set: Operation not permitted! Message-ID: <20220228175835.624eb178@hermann> In-Reply-To: <20220228171127.2469a57d.grembo@freebsd.org> References: <20220228161545.251fe0d8@hermann> <20220228171127.2469a57d.grembo@freebsd.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-UID: 27bc31 X-Rspamd-UID: 717799 X-Rspamd-Queue-Id: 4K6mmf3Z24z3NvN X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=walstatt-de.de header.s=DKIM001 header.b=j6i2gRLI; dmarc=none; spf=none (mx1.freebsd.org: domain of freebsd@walstatt-de.de has no SPF policy when checking 85.220.129.31) smtp.mailfrom=freebsd@walstatt-de.de X-Spamd-Result: default: False [-2.90 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[walstatt-de.de:s=DKIM001]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[walstatt-de.de]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[walstatt-de.de:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; SUBJECT_ENDS_EXCLAIM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current,freebsd-virtualization]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; MID_RHS_NOT_FQDN(0.50)[]; ASN(0.00)[asn:25394, ipnet:85.220.128.0/17, country:DE]; FREEMAIL_CC(0.00)[freebsd.org,gmail.com]; RCVD_TLS_ALL(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[85.220.129.31:from] X-ThisMailContainsUnwantedMimeParts: N On Mon, 28 Feb 2022 17:11:27 +0100 Michael Gmelin wrote: [...] schnipp [...] > > > > poudriere jail -l: > > > > # poudriere jail -l > > JAILNAME VERSION ARCH METHOD TIMESTAMP PATH > > 123-amd64 12.3-RELEASE amd64 > > url=https://download.freebsd.org/releases/a ... 3-RELEASE/ 2022-02-24 > > 14:14:25 /mnt/poudriere/jails/123-amd64 130-amd64 13.0-RELEASE amd64 > > url=https://download.freebsd.org/releases/a ... 0-RELEASE/ 2022-02-24 > > 14:11:32 /mnt/poudriere/jails/130-amd64 > > > > The jail.conf for this specific jail is as follows: > > > > [...] > > pulverfass-001 { > > devfs_ruleset = 13; > > enforce_statfs = 1; > > exec.clean; > > exec.consolelog = > > /mnt/extensions/bastille/logs/pulverfass-001_console.log; exec.start > > = '/bin/sh /etc/rc'; exec.stop = '/bin/sh /etc/rc.shutdown'; > > host.hostname = XXXXXXXXX; > > mount.devfs; > > mount.fstab = /mnt/extensions/bastille/jails/pulverfass-001/fstab; > > path = /mnt/extensions/bastille/jails/pulverfass-001/root; > > securelevel = 0; > > > > vnet; > > vnet.interface = e0b_bastille4; > > exec.prestart += "jib addm bastille4 igb0"; > > exec.prestart += "ifconfig e0a_bastille4 description \"vnet host > > interface for Bastille jail pulverfass-001\""; exec.poststop += "jib > > destroy bastille4"; > > > > allow.mount; > > allow.mount.fdescfs; > > allow.mount.devfs; > > allow.mount.tmpfs; > > allow.mount.nullfs; > > allow.mount.procfs; > > allow.mount.linsysfs; > > allow.mount.linprocfs; > > allow.mount.zfs; > > > > allow.chflags; > > allow.raw_sockets; > > allow.socket_af; > > allow.sysvipc; > > > > linux = new; > > > > exec.created += "/sbin/zfs jail ${name} BUNKER00/poudriere"; > > exec.start += "/sbin/zfs mount -a"; > > exec.poststop += "/sbin/zfs unjail BUNKER00/poudriere"; > > > > } > > [...] > > > > Tracking the execution of the build process by issuing > > > > poudriere -x bulk ... > > > > and examin the resulting trace doesn' tgive me any hint, the error > > reported above immediately occurs when the jail is about to be > > started: > > > > + set -u +x > > + jail -c persist 'name=123-amd64-head-default' > > 'path=/mnt/poudriere/data/.m/ \ 123-amd64-head-default/ref' > > 'host.hostname=basehost.local.domain' \ 'ip4.addr=127.0.0.1' > > 'ip6.addr=::1' allow.chflags allow.sysvipc jail: jail_set: Operation > > not permitted > > + exit_handler > > [...] > > > > Searching the net revealed some issues with setting IP4 and IP6 in > > poudriere, but those findings are dated back to 2017 and 2014 and I > > guess this is solved right now. > > > > The difference between our manually jail.conf driven setup and the > > XigmaNAS/bastille based one is, bastille uses jib/netgraph based > > seutups of the vnet and the ip4/ip6 is setup from rc.conf, while we > > use epair in the other world and the ip is setup from withing the > > jail definition in jail.conf. > > > > I'm out of ideas here and after two days of trial and error and > > trying to understand what's going on lost ... Any hints or tipps? > > > > Thanks in advance, > > > > O. Hartmann > > Hi Oliver, > > I don't see `children.max` set in any of the configuration you shared > above. > > Cheers > Michael > Hello Michael, bummer! I was so selfconfident because I copied the initial config from a working test and had this attribute already set that I never checked again its existence - and started reorganizing the jail.conf attributes ... A fine observation and a full hit: after setting children.max= 128; the poudriere jail started working ... didn't wait for the finish so far. I'm sorry for the noise - thanks for you eyes ... Kind regards, Oliver