From nobody Tue Dec 13 23:03:42 2022 X-Original-To: freebsd-jail@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NWvF02KBzz4jnnQ for ; Tue, 13 Dec 2022 23:03:48 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NWvF01ZS9z3tkq for ; Tue, 13 Dec 2022 23:03:48 +0000 (UTC) (envelope-from bz@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1670972628; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=T9mvYqF2fYGdBfTOLloM25RVIXK3aUjtKbeZy+G1YOo=; b=asnguyhnSzX0QJOV9pVtrLR3hrS/GyCZ/CZBraFYyKPtk1m9cyuINEV7JCXt7g6+fO4OhW UH0mo/zu5j3eBi5Pen5fz6hOBW5Z6GK8SW2hdTb7uCzQSSr9URxSwFNEZzbDwg/7/bU7DS 25yATmXeeiC5R+bxmo+bamGYtA8KKQpdDkLqF3Pj9vTyeGO3VJKLuy03YpdPXASSZ89W/B dYuiuMrt+/k51Uq6xGoNwZxbNgUKAECRSeftzQwxEVFdopBj9ntqsZ7aSzHcZaP6XbSJcL 5RNI4Av2EK/e8tfT8bcC+2CfHEWjR4uawBPs/eGYjsNPlArvQM8XHVSrq28XwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1670972628; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=T9mvYqF2fYGdBfTOLloM25RVIXK3aUjtKbeZy+G1YOo=; b=G00zl57zhCztCvI5+1Tlz/sN1LJHhh5FMfWWD3aOYou2F5N36UYqb+wqoOrhvu2387ryMm zmMinZ2pn0DO5WqfkqOvfIqVVmrcFJOmRs2aCafvN8AEcFmjD9e2/u8p3A8eWSjIkNWywP /W0UvnA98RDy6PS4+yOsi9lylggOcV/RTAZGSNbquNDu+ly137ovRcJEafdBEj5U17ciZg XOcsDYeYk2Hb5WW5/weLA8P/4S9Cs4RYpRxqKQS/OkSemh2Nw1RXGPr8UlvVyO3ivsRTwP iAQ2Wcb5YGIefeEEXZR53HpurtqxHVKQNolKKLOPadxj0Sbw7Zi99g9RBT4V4g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1670972628; a=rsa-sha256; cv=none; b=OKpW/7SQWzpaaOodnDreV+0qX1zLwRfiotLhVEL/bOAXVz4XN8hlNad6eLBVyoSRmIotlH zvyX7sCvOwGrZkmrQcubReWNp6uHVQA3qg44c0ed0Opci9amtquW3HuW21kKh2/p6rbK2h xB6Deq+kdHw4XVB3+HTdifWndnuHmgGT1GSsaZaqbo13XTYuNj8NMeg9uKKrKodptAltLN /Ytb70IB4rIkasCp1dwCLZR9cPxIPaxJlqpj0MLAbMQEw/daW9boPc/8QeC/jFgFe8WFx8 S6UywsRS3y25ezOJrzAynqcJ0vuEpOKS7ddZt7FdlddHxpk17pLKNt7psZ+AJQ== Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:13b:39f::9f:25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) (Authenticated sender: bz/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4NWvDz6ZCJzRKQ for ; Tue, 13 Dec 2022 23:03:47 +0000 (UTC) (envelope-from bz@FreeBSD.org) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:1025]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id D026B8D4A129 for ; Tue, 13 Dec 2022 23:03:45 +0000 (UTC) Received: from content-filter.t4-02.sbone.de (content-filter.t4-02.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:2742]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 429655C3A833 for ; Tue, 13 Dec 2022 23:03:45 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:4902:0:7404:2:1025]) by content-filter.t4-02.sbone.de (content-filter.t4-02.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:2742]) (amavisd-new, port 10024) with ESMTP id GFoyLQsKt5LT for ; Tue, 13 Dec 2022 23:03:43 +0000 (UTC) Received: from strong-iwl0.sbone.de (strong-iwl0.sbone.de [IPv6:fde9:577b:c1a9:4902:b66b:fcff:fef3:e3d2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 5993A5C3A830 for ; Tue, 13 Dec 2022 23:03:43 +0000 (UTC) Date: Tue, 13 Dec 2022 23:03:42 +0000 (UTC) From: "Bjoern A. Zeeb" To: freebsd-jail@FreeBSD.org Subject: What's going on with vnets and epairs w/ addresses? Message-ID: <5r22os7n-ro15-27q-r356-rps331o06so5@mnoonqbm.arg> X-OpenPGP-Key-Id: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 List-Id: Discussion about FreeBSD jail(8) List-Archive: https://lists.freebsd.org/archives/freebsd-jail List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-jail@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset=US-ASCII X-ThisMailContainsUnwantedMimeParts: N Hi, I have used scripts like the below for almost a decade and a half (obviously doing more than that in the middle). I haven't used them much lately but given other questions I just wanted to fire up a test. I have an end-November kernel doing the below my eapirs do not come back to be destroyed (immediately). I have to start polling for the jid to be no longer alive and not in dying state (hence added the jls/ifconfig -l lines and removed the error checking from ifconfig destroy). That seems sometimes rather unreasonably long (to the point I give up). If I don't configure the addresses below this isn't a problem. Sorry I am confused by too many incarnations of the code; I know I once had a version with an async shutdown path but I believe that never made it into mainline, so why are we holding onto the epairs now and not nuking the addresses and returning them and are clean? It's a bit more funny; I added a twiddle loop at the end and nothing happened. So I stop the script and start it again and suddenly another jail or two have cleaned up and their epairs are back. Something feels very very wonky. Play around with this and see ... and let me know if you can reproduce this... I quite wonder why some test cases haven't gone crazy ... /bz ------------------------------------------------------------------------ #!/bin/sh set -e set -x js=`jail -i -c -n jl host.hostname=left.example.net vnet persist` jb=`jail -i -c -n jr host.hostname=right.example.net vnet persist` # Create an epair connecting the two machines (vnet jails). ep=`ifconfig epair create | sed -e 's/a$//'` # Add one end to each vnet jail. ifconfig ${ep}a vnet ${js} ifconfig ${ep}b vnet ${jb} # Add an IP address on the epairs in each vnet jail. # XXX Leave these out and the cleanup seems to work fine. jexec ${js} ifconfig ${ep}a inet 192.0.2.1/24 jexec ${jb} ifconfig ${ep}b inet 192.0.2.2/24 # Clean up. jail -r ${jb} jail -r ${js} # You want to be able to remove this line ... set +e # No epairs to destroy with addresses configured; fine otherwise. ifconfig ${ep}a destroy # echo $? # Add this is here only as things are funny ... # jls -av jid dying # ifconfig -l # end ------------------------------------------------------------------------ -- Bjoern A. Zeeb r15:7