From nobody Sun Dec 18 16:52:58 2022 X-Original-To: freebsd-jail@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NZpn24ZP0z1GQLl for ; Sun, 18 Dec 2022 16:53:10 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NZpn22FBDz4CMs; Sun, 18 Dec 2022 16:53:10 +0000 (UTC) (envelope-from kevans@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1671382390; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=+cn0thUASzpf2IzuuHtGIFqMyI8S51i7W6WMfVayoiI=; b=K0nHrm1YtEUtGLTpAXzc5TVkxrVNvoqf6B3idAE9LH00TlajNTMIf5OhJZN1P7L4yZHxVm 3A4YSvDTiRl+ffkEZU7l5INbs74hNvx0s+TPK7QI2k0ZWtTeHb+zFJzXy+svqkQIjaiA1+ U0tYupia+H90wRA6OrdP4P/TMiMWMJk76c9XaZTnSkF4jz8tBMNPcREEAnFvNY1QblDNFc ADMLg19i23Y2cNg3o5Aku4yxrspFN3dMTH7bMtpLoxMXiKw60ikUQq9D+f6TcOb3lj+QiB 10Be8OsIIv48UOiG+aKiSpW1KuamTP9+E/60AEY5/4QLrRmwlmUYpyoKwBiqAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1671382390; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=+cn0thUASzpf2IzuuHtGIFqMyI8S51i7W6WMfVayoiI=; b=nfUuMVZY/TiZMxy3Cv8Vd/96BV7sh41FSa1656bHrIDoTi060NWC9o38wmmn0zDtRhnQYW Lm8DbTJQC+waiAblYm1/R1zGw+HfzkCXvAfTf/BI+YFwJYhHUsT7OvLsrWicDLOtOwRE+l CqPJYifPpM9pzzA7VhdUdJ7ftnuxQVIiuGwBt24f7mdDOi6pUdj34Nr9Cwl54MGwR96QJj L5cvgeH0BFpMGaVVMIq549rQe0quQdu4itNiy84P4ZrBFaTgVxwVuF0xEO5Gddj4T7Cora uI2jRSV4lHq/q5i2YhKQFaZyJ30o8sDKzFVYWvVuJLuNO1kAScpUUO0T+zTR4Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1671382390; a=rsa-sha256; cv=none; b=Di4c4IQfohirFgYzQvWNfjKmFziKZe+D7a+5YPZocrsX9jhjMHfMtEFMlFKUxw3MmHGLX+ V3ZbP1bY3ICFRozumEnhpQyqrBgXDgyVbDU9WsNhDSPxc5Qf+iJteDzUQ/Ex1QZ/dbnU5w bHsD8/gm0TNh6WfFzgC/UY6uXO9MdT+sKPk8URTao6fPrlZNGwGW31jmB/2Ri3gXyZerts qcvpHefaDMHgGVwzpjMGB3pATm+NrzcKejydo9l78tiM9Tv00diN/fdf9YqwLVr1RwTTTH x8wrjOueWFohG2vnSwoz3i8I1wS892D2/GYdlbrSrrUEo4RJl3lK6Bl9Ra1q/g== Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id 4NZpn21BybzJtM; Sun, 18 Dec 2022 16:53:10 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-qk1-f181.google.com with SMTP id h8so2841487qkk.8; Sun, 18 Dec 2022 08:53:10 -0800 (PST) X-Gm-Message-State: ANoB5pmXCPIFvwAwPNocyLJ/2WIYEb1/RND3/0ORa0ZWjkjP87JghrWc KPc0C2G1+O3teD6jq6jZD9kD90SsKCR4SHHYqLI= X-Google-Smtp-Source: AA0mqf4KJ9MLDkSudKNx1hP6ANkk2uqPqB4dq0IuwFVysDNpxpeMWrrbYpd8noBS9X0tq62W0dx0eUIZ4SclAxHcdSA= X-Received: by 2002:a05:620a:2f8:b0:6ff:8af3:c97f with SMTP id a24-20020a05620a02f800b006ff8af3c97fmr1540351qko.425.1671382389650; Sun, 18 Dec 2022 08:53:09 -0800 (PST) List-Id: Discussion about FreeBSD jail(8) List-Archive: https://lists.freebsd.org/archives/freebsd-jail List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-jail@freebsd.org MIME-Version: 1.0 References: <5r22os7n-ro15-27q-r356-rps331o06so5@mnoonqbm.arg> <150A60D6-6757-46DD-988F-05A9FFA36821@FreeBSD.org> In-Reply-To: From: Kyle Evans Date: Sun, 18 Dec 2022 10:52:58 -0600 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: What's going on with vnets and epairs w/ addresses? To: Gleb Smirnoff Cc: Zhenlei Huang , "Bjoern A. Zeeb" , "freebsd-jail@freebsd.org" , Mark Johnston Content-Type: text/plain; charset="UTF-8" X-ThisMailContainsUnwantedMimeParts: N On Sat, Dec 17, 2022 at 11:22 AM Gleb Smirnoff wrote: > > Zhenlei, > > On Fri, Dec 16, 2022 at 06:30:57PM +0800, Zhenlei Huang wrote: > Z> I managed to repeat this issue on CURRENT/14 with this small snip: > Z> > Z> ------------------------------------------- > Z> #!/bin/sh > Z> > Z> # test jail name > Z> n="test_ref_leak" > Z> > Z> jail -c name=$n path=/ vnet persist > Z> # The following line trigger jail pr_ref leak > Z> jexec $n ifconfig lo0 inet 127.0.0.1/8 > Z> > Z> jail -R $n > Z> > Z> # wait a moment > Z> sleep 1 > Z> > Z> jls -j $n > Z> > Z> After DDB debugging and tracing , it seems that is triggered by a combine of [1] and [2] > Z> > Z> [1] https://reviews.freebsd.org/rGfec8a8c7cbe4384c7e61d376f3aa5be5ac895915 > Z> [2] https://reviews.freebsd.org/rGeb93b99d698674e3b1cc7139fda98e2b175b8c5b > Z> > Z> > Z> In [1] the per-VNET uma zone is shared with the global one. > Z> `pcbinfo->ipi_zone = pcbstor->ips_zone;` > Z> > Z> In [2] unref `inp->inp_cred` is deferred called in inpcb_dtor() by uma_zfree_smr() . > Z> > Z> Unfortunately inps freed by uma_zfree_smr() are cached and inpcb_dtor() is not called immediately , > Z> thus leaking `inp->inp_cred` ref and hence `prison->pr_ref`. > Z> > Z> And it is also not possible to free up the cache by per-VNET SYSUNINIT tcp_destroy / udp_destroy / rip_destroy. > > This is known issue and I'd prefer not to call it a problem. The "leak" of a jail > happens only if machine is idle wrt the networking activity. > > Getting back to the problem that started this thread - the epair(4)s not immediately > popping back to prison0. IMHO, the problem again lies in the design of if_vmove and > epair(4) in particular. The if_vmove shall not exist, instead we should do a full > if_attach() and if_detach(). The state of an ifnet when it undergoes if_vmove doesn't > carry any useful information. With Alexander melifaro@ we discussed better options > for creating or attaching interfaces to jails that if_vmove. Until they are ready > the most easy workaround to deal with annoying epair(4) come back problem is to > remove it manually before destroying a jail, like I did in 80fc25025ff. > It still behaved much better prior to eb93b99d6986, which you and Mark were going to work on a solution for to allow the cred "leak" to close up much more quickly. CC markj@, since I think it's been six months since the last time I inquired about it, making this a good time to do it again... Thanks, Kyle Evans