From nobody Tue Dec 13 23:54:17 2022 X-Original-To: jail@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NWwML07tZz4k956 for ; Tue, 13 Dec 2022 23:54:22 +0000 (UTC) (envelope-from bz@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NWwMK6lFRz40qm; Tue, 13 Dec 2022 23:54:21 +0000 (UTC) (envelope-from bz@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1670975661; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=qJbTLqqkjA8jg9py/M2ZKo8RlIVfJG9uDiOf60vh8Qc=; b=Tv1TFKeVq4HKq6Lu8U8rzj000O64ziZ7aZW1EZCDsfzWNDTo9XsBiYkkJRF6x+x0swHROw x8QRHwVL1Qjk4KTCdaS11Z0VgPeu5w6t5xc/pqekIXS9gzBz2h7K6ISlLwMr61GXZPT02q chkTPCkqEwl5ZbPxyLXLh/cyE5h45FCRm07spedrD+4SQAbWLqWy1SXgvNiliq11ZFVlBy QdkbE8TqsbCuA2NUiGjbIlm4rwx6dZ2RoD0QNCMTJ7srM6O36KS67cbHJN2X7e2958KgYf wEGI1ZJCtk5VeTNNJZw+4pLVTGW8JL3VCWZ14hmnukvUO4bprkduufmbrnhK3w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1670975661; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=qJbTLqqkjA8jg9py/M2ZKo8RlIVfJG9uDiOf60vh8Qc=; b=dNIDfw3RnHrW0CCvFN+QPxvTV4bz6WTXNP6/1eFP+bW7utXonFdo8ENBea0t8b3nASl3OJ ZLoduMkBWq/+ejM48yLWYRBs9DqpSaqK13FlEvYyeZkpNyyC33DRcW+30gF9aPd+aUKiFh SomcMCIO9dnSqFMg/AiexvWNAnap8jPbUjX4pEK7lSSctp7viIqu15wKQqtRs1RBRQTXml gdvI4claUKHYMWzPPJcSZaBCazTIFW4uxhPfan4P6WTOnBOtv61wKEEHrUs6HZ1qJWTcxn J7PF8MSXMLSdFDhoyvV0CVA9R4LGm5OgjVAc2BjjLlTFvTIS+bX75WoSYoN5pA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1670975661; a=rsa-sha256; cv=none; b=egzQD3/Gb5Wb9OjCq6H5zDVgmNv0OpkInl1s+aPghd7CxS5+tsl9u+Cn6u8t/M3krl9y9z teG0D4Z6tUDf5CLkCH5MjAE9WAuQO05XE2BIMJtoZWlHDH2BWk90xnnoZEXY4E2DBKTXfD DKEwG9e1dUSK667Gxr95JKCGaeFqP/OoUIcF0TyLJPQ38lDuJGsjocGUFHpAIsmYFuiNCN p20W39ixx3MKTtiRUbocJ38/E3j1xfjlXvBIqkuplfx1gfep6y1ioE7/aNur0vAdkcpGCE ShdxE91Wdd4R9wYvj3NolZs8qjeovcVbcchp4eyxH/X28sXSM3jchlSdjmAGiA== Received: from mx1.sbone.de (cross.sbone.de [195.201.62.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) (Authenticated sender: bz/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4NWwMK55HDzSbH; Tue, 13 Dec 2022 23:54:21 +0000 (UTC) (envelope-from bz@freebsd.org) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:1025]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 76DD08D4A162; Tue, 13 Dec 2022 23:54:20 +0000 (UTC) Received: from content-filter.t4-02.sbone.de (content-filter.t4-02.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:2742]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 2301C5C3A876; Tue, 13 Dec 2022 23:54:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:4902:0:7404:2:1025]) by content-filter.t4-02.sbone.de (content-filter.t4-02.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:2742]) (amavisd-new, port 10024) with ESMTP id UbHAl3xdLpNU; Tue, 13 Dec 2022 23:54:18 +0000 (UTC) Received: from strong-iwl0.sbone.de (strong-iwl0.sbone.de [IPv6:fde9:577b:c1a9:4902:b66b:fcff:fef3:e3d2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id F1DB65C3A830; Tue, 13 Dec 2022 23:54:17 +0000 (UTC) Date: Tue, 13 Dec 2022 23:54:17 +0000 (UTC) From: "Bjoern A. Zeeb" To: Andrew Gallatin cc: "pjd@FreeBSD.org" , James Gritton , jail@freebsd.org, "glebius@FreeBSD.org" Subject: Re: prison_flag() check in hot path of in_pcblookup() In-Reply-To: Message-ID: <6r10qop4-7p83-qs6s-q3r0-64756n243rp5@serrofq.bet> References: <6on81os3-501-s5n2-8nos-p85n8op23232@serrofq.bet> X-OpenPGP-Key-Id: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 List-Id: Discussion about FreeBSD jail(8) List-Archive: https://lists.freebsd.org/archives/freebsd-jail List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-jail@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset=US-ASCII X-ThisMailContainsUnwantedMimeParts: N On Tue, 13 Dec 2022, Andrew Gallatin wrote: > [ I added pjd, since the original patch came from him ] > > Just to make sure I understand, I have a simple yes/no question: > > Can jails and the host ever share the same (local) port and the same IP? Can they currently (I tested only for TCP)? - local binds can overlap like they can with just the base system. so bind(... {AF_INET, laddr, lport} ... ) works fine (REUSEPORT). - tcp connect of a 2nd socket to the same {faddr, fport} from the above bind will fail with 'Address already in use' [currently] [I believe that would mean your patch could go in? Where does the error come from [%]?] [*] - tcp listen will work on {laddr, lport} if run in paralllel (REUSEPORT) or in base and jail at the same time. [%] likely in_pcbconnect_setup() ? Also one should check the other order (jail first then base); also we assume no other race conditions in this rather simple testing... [*] Now someone should run this on a FreeBSD 7.3 / 8.x or later and see how it behaves as the stack might have behaved differently. Also if you have two physical machines or two VMs connected remove the VNET layer and just (manually) test the two parts firing up one extra jail on each base system. I just used vnets for simplicity of my testing. (sorry vnet cleanup currently screwed as it seems 15 years of working also have changed due to other changes; you cal always run jail -r jl jr manually). I haven't done user space socket programming in a while so this was fun (and hopefully does what it should for this test case). I put the simple sources (shell script and C file) up at: https://people.freebsd.org/~bz/tmp/jail-in_pcblookup/ HTH /bz + pwd + STESTBIN=/home/test/socket + PORT=7 + jail -i -c -n jl 'host.hostname=server.example.net' vnet persist 'children.max=1' + js=211 + jail -i -c -n jr 'host.hostname=base.example.net' vnet persist 'children.max=1' + jb=212 + jexec 211 ifconfig lo0 inet 127.0.0.1/8 alias up + jexec 211 ifconfig lo0 inet6 ::1/128 alias + jexec 212 ifconfig lo0 inet 127.0.0.1/8 alias up + jexec 212 ifconfig lo0 inet6 ::1/128 alias + ifconfig epair create + sed -e 's/a$//' + ep=epair102 + ifconfig epair102a vnet 211 + ifconfig epair102b vnet 212 + jexec 211 ifconfig epair102a inet 192.0.2.1/24 + jexec 212 ifconfig epair102b inet 192.0.2.2/24 + jexec 211 jail -i -c -n jsj 'host.hostname=jails.example.net' 'ip4.addr=192.0.2.1' persist + jexec 211 /home/test/socket 192.0.2.1 7 /home/test/socket pid 23254 listening on [192.0.2.1 7] + jsj=213 + echo 'Listing listening connections from the server (base) system' + jexec 213 /home/test/socket 192.0.2.1 7 Listing listening connections from the server (base) system + jexec 211 netstat -an /home/test/socket pid 23257 listening on [192.0.2.1 7] Active Internet connections (including servers) Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp4 0 0 192.0.2.1.7 *.* LISTEN tcp4 0 0 192.0.2.1.7 *.* LISTEN + jexec 212 jail -i -c -n jbj 'host.hostname=jailb.example.net' 'ip4.addr=192.0.2.2' persist + jbj=214 + sleep 1 + echo 'Starting connection from base jail' Starting connection from base jail + sleep 1 + jexec 212 /home/test/socket 192.0.2.2 12345 192.0.2.1 7 /home/test/socket pid 23257 accepted [192.0.2.2 12345] /home/test/socket pid 23261 [192.0.2.2 12345] sleeping 60. + echo 'Starting connection from plain-old IP jail' Starting connection from plain-old IP jail + sleep 1 + jexec 214 /home/test/socket 192.0.2.2 12345 192.0.2.1 7 socket: connect : Address already in use + echo 'Listing server connections from the server (base) system' Listing server connections from the server (base) system + jexec 211 netstat -an Active Internet connections (including servers) Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp4 0 0 192.0.2.1.7 192.0.2.2.12345 ESTABLISHED tcp4 0 0 192.0.2.1.7 *.* LISTEN tcp4 0 0 192.0.2.1.7 *.* LISTEN + echo 'Listing client connections from the base system' Listing client connections from the base system + jexec 212 netstat -an Active Internet connections (including servers) Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp4 0 0 192.0.2.2.12345 192.0.2.1.7 ESTABLISHED + sleep 60 ^C -- Bjoern A. Zeeb r15:7