From nobody Mon Mar 24 06:33:36 2025 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZLjtn4JkHz5rTf6; Mon, 24 Mar 2025 06:33:53 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZLjtn3flVz3v9Q; Mon, 24 Mar 2025 06:33:53 +0000 (UTC) (envelope-from kp@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1742798033; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nEXwySIoNlBOQXpauglcM1w5Kg8OF5mv0X44SOh+Da0=; b=klb7cHUpcbegKysGMeN9GCZ3DJP4Mi9QbzrZGkEdMM73Ck4XZCxLVdloXHdBl9qkg2RIdQ v56ud5VHHFr8uVE12d2DrFTH8wK0dIgpAJVL0YznH7YFABSNK6wZrGzWrIR7dkCC3Ef5D3 S9RHsN1ML9hlon621ip/hwyZt7wJThsSougy/uGg8NoKn7k84gRkFz8BynxqBQHTA0SeZn O3itRZOh6YxmXTo9wpbGgL/jTg7JEaKljW1FvBoPoV8UTe51qqK9AWTx3BeeNw6KmPurUW wItbpKG91rc1NIsHMIEckrqIOVHBBwfEFBNrK11TClewC8ITGBDxE6XNZM7iaA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1742798033; a=rsa-sha256; cv=none; b=WIfjaSX98W1AJwgM3YTxW2MAE2H+X0gv1NGsT+NcGrC7c1yv3Tc8TTGzFvUAFn9JXVECQ8 dKDYUvnotKNc6Cy7lG1ZT0BfO4YvC7wEZFUcMB9qsrDUdWWXp//jeXYuiwhMVApO24wVVf JX/6I3j9nnRXejGIYfDgyPNZYNgek2kU/ItGXFXTwJy7r9NsDGCY/nZf/lm/cTsJvdaSom tV2q33u9/xcV2dPo6FolkHHUxuucu+Sf2bbB9V9AjTzuxl4pdl0SW3+IiPoaxqwO/LYiCB LeBE/X5RA+p5s/oiRXn9YLeT5cWYkSGI6veIHfxZPHEjARlWCbfYzhmMFdXp6Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1742798033; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nEXwySIoNlBOQXpauglcM1w5Kg8OF5mv0X44SOh+Da0=; b=YzjnHm7ZMCswG0Utdi7y4d1Mcf/hVj84EY61pav4oImRm+PwT+2p0EM7mSkSnX3LRQ1waB PYrx+hjTSm2+iSdn+AKcxJdK0Sa8luOsrFfiAqtU7PmAr+aGjC3wvuadG+5LHnVR8BykjN xg7AJkgFVLTcrSYbHcjGYU5aJO7VQy+r3SIMZbFyJvtTOlSzd+HIfNwBarFltdwAx9+XW4 XgeXsUyb88/0aXAH4ndOKyXbJvC3eYC9NjnYmFPxXS+0aSEr2emY0o3L0/eeR9DZT6CaNL knwUWJ5YO9AaUprIzMyIkTJ1aG6BRbzexvAy8N/YALOiqBQCChYO8cC4U8pbtg== Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mx1.codepro.be", Issuer "R10" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id 4ZLjtn2MPJz25S; Mon, 24 Mar 2025 06:33:53 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: by venus.codepro.be (Postfix, authenticated sender kp) id 7CA861EF21; Mon, 24 Mar 2025 07:33:44 +0100 (CET) From: Kristof Provost To: Gleb Smirnoff Cc: igoro@freebsd.org, src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: 0849f1634a70 - main - tests/netinet: add test for IP_MULTICAST_IF Date: Mon, 24 Mar 2025 15:33:36 +0900 X-Mailer: MailMate (2.0r6222) Message-ID: In-Reply-To: References: <202503222340.52MNekCX071219@gitrepo.freebsd.org> <7440BE85-66E8-4D1A-AC15-8B944F5C2951@FreeBSD.org> List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; markup=markdown Content-Transfer-Encoding: quoted-printable On 24 Mar 2025, at 15:12, Gleb Smirnoff wrote: > On Mon, Mar 24, 2025 at 11:45:55AM +0900, Kristof Provost wrote: > K> > tests/netinet: add test for IP_MULTICAST_IF > K> > > K> It bugs me a little that we have to jump through hoops to get a C pr= ogram > K> running in a specific jail/network setup. > K> It=E2=80=99s somewhat common for that to happen, and I=E2=80=99m sur= e there=E2=80=99d be more cases > K> if it were easier to do (e.g. the getaddrinfo tests you=E2=80=99ve b= een working on > K> could be even more useful with a custom DNS server, so in their own = vnet > K> jail). > > It bugs me, too. :) Igor has already pushed changes that allow you to s= pecify > in the Makefile: > > TEST_METADATA.foo+=3D "execenv=3D"jail" execenv_jail_params=3D"vnet" > The primary intent behind that was to allow for parallel test execution. A lot of tests, and a lot of this is my fault, use the same network range= s and jail names, so if we run them at the same time they=E2=80=99d confl= ict and break in unexpected ways. Running them inside a vnet jail avoids this issue. I was actually looking at adding this to your multicast tests, but you do= n=E2=80=99t assign addresses on the host=E2=80=99s epair, so there=E2=80=99= s not much point. > Then this gets into Kyuafile after build. This is a step forward, but n= ot > enough. First problem is that the created jail is empty and doesn't hav= e any > interfaces. Second problem is that the specified environment affects al= l tests > inside one test file. > > Ideally we'd like to have a declarative language to describe jail envir= onment > for a test. This is basically what vnet.subr does for shell tests and = what > atf_python.sys.net.vnet does for python tests, but brought to kyua leve= l and of > course agnostic to the language of a test. Igor is thinking in this di= rection > and hopefully will eventually deliver something. > Interesting idea. I fear it would be difficult to get something that=E2=80= =99s expressive enough to cover all the cases we care about, while still = being simple enough to be easy to use. On the other hand, perhaps we don=E2=80=99t need something that covers 10= 0% of cases. We=E2=80=99re probably never going to be able to build the s= orts of complex setups (for values of complex that are =E2=80=98three jai= ls with routes, a simple daemon in one or more and some pf rules) I often= build for pf tests, but perhaps we can cover 80% of cases with a simple = =E2=80=9CBuild a jail with one interface with a known address and a defau= lt route=E2=80=9D. Perhaps a =E2=80=98execenv_jail_init=3Dfoo.sh=E2=80=99 calling a separate= script to do the setup? > K> The downside is that we=E2=80=99re compiling the C test code on ever= y run, but I > K> expect test programs to be tiny, so that=E2=80=99s not too much of a= cost. > K> > K> Does this seem useful to you too? > > That would fail on decreased installation, that has tests, but doesn't = have > compiler. This is not just a made up problem, this is what many would = do in > CI, cause you want usually as quick as possible response time from CI, = and if > your team is not hacking the compiler, you won't add compiler to the CI= build. > That=E2=80=99s a good point, and not something I think we can reasonably = do from atf-sh. Not unless we do a pre-processing step during =E2=80=98ma= ke install=E2=80=99 (on top of the =E2=80=98add the shebang line=E2=80=99= we already do) to extract C code, build it and install it somewhere the = test can find it. I think I don=E2=80=99t dislike the current approach enough to try that. Best regards, Kristof