From nobody Thu Mar 19 15:47:07 2026 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4fc97F6Sndz6VXWv for ; Thu, 19 Mar 2026 15:47:21 +0000 (UTC) (envelope-from zlei@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R13" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4fc97F69vGz3VZ2; Thu, 19 Mar 2026 15:47:21 +0000 (UTC) (envelope-from zlei@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1773935241; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=NZLssMHayF6ax3Y/h0YEyVJ8j16xBR2LHsYg6sHmST8=; b=tlFMnZuAGVoEpUYvGHwGACcbt+dA9M3jsS2lOCHRclUPXEKI3Maaw1rrGc+pJ+zSH1nfkf VYteyHzcflv5CkSCQGeQ0Zo0/yHW++jAyfkQxnG5IA0QQb2i7OtE8F3sfuPQpEBSdnqAG/ 2GhwUWvTqJFmuqwCsuGsgQ9YvYr36Hw18Wuo/wacIIlGf7fmi9xL7GtI4xnBuiTF+A6Y6h WEeAqGEtFqI9YuT/cdLP+VyFs0kfnbpPzA+FrHkxopzgj3VctS8UN/79NGAGFYCurpGMRa K4XFm2BplE6BSIQZe3WseZjt/7hQMvlqimyeZOMOQsObnBFANS8UfoJrHHr1zw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1773935241; a=rsa-sha256; cv=none; b=U2EdElNJcBlISSQTr1dEfCjMgmIR+D2iHQc/wnkyKYNxeocmrxId7SIkU5pvPYVvZi2LQD qs+LpguFMd+9VdfM3oq2eDrdFgnD30U1aTTr86ff+l7vY3rPgarG3Rlg1qjzD379mwDz4s Iin7UYVpdCp06JKTlFIKrwavFZ0jpQCxMOKmEm8GcjFyUdNEH4yoe2Acf2UiFZxAkLnZh5 DjcTGIW1FLfg9uMrneNqKjihjkWrXfuItPCpod4u6yRfY3BRMRQsB2s6lT+MezDtCuHZGO Qy8MR8BPJFIdtuHxrAiPCQOg6idF+GlOgemIOC3Ygs10jUFo0+uOSfN8D1aQ0g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1773935241; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=NZLssMHayF6ax3Y/h0YEyVJ8j16xBR2LHsYg6sHmST8=; b=UhVLeNjGLxNF5TmDpBjZtsPWx0VJqgTdlpRMcSbtckn5o0efvJS13K6LCsspis04yQxnGB BYp/pYex2Od+Crb8Jg+rIN2Wrn6W1IZeOgHUBRm/Cyc/nHkC6YIwQU4LuFsX8/zyPgRTky 5x6SCCkMqnPcmtGKl4PjR7RcjOZHEMK5q7023NznLYWaWLNvTE/hm78K5ETU7hmxRmaaCb 9vt9+i+jHf8asumTFqPbTBmmpogYyG+/DnchFUAHtMskQfusNVdFutYAyTby9JbaahwkfN 5KJ+7O/ae19ASeBB1x9GeUp5/GWuxWiJz3SXwzmXA2fJzsTl+t376NDgRHmTxg== Received: from smtpclient.apple (ns1.oxydns.net [45.32.91.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: zlei/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4fc97D4MdMzs0j; Thu, 19 Mar 2026 15:47:20 +0000 (UTC) (envelope-from zlei@FreeBSD.org) From: Zhenlei Huang Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_17AB9DFF-2CE9-4D2B-8EC7-0CC734F90566" List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.10\)) Subject: Re: userret: returning with the following locks held Date: Thu, 19 Mar 2026 23:47:07 +0800 In-Reply-To: Cc: FreeBSD Current To: Konstantin Belousov References: X-Mailer: Apple Mail (2.3696.120.41.1.10) --Apple-Mail=_17AB9DFF-2CE9-4D2B-8EC7-0CC734F90566 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Mar 19, 2026, at 2:01 AM, Konstantin Belousov = wrote: >=20 > On Wed, Mar 18, 2026 at 05:36:00PM +0800, Zhenlei Huang wrote: >> Hi, >>=20 >> While I'm working on the if_detach() / if_vmove() racing issue and I = wrote >> a test and try to repeat the race condition and got this assert = panic, >>=20 >> ``` >> <6>epair1b: link state changed to UP >> userret: returning with the following locks held: >> exclusive sx ifnet_detach_sx (ifnet_detach_sx) r =3D 1 = (0xffffffff83ab75e0) locked @ /home/zlei/freebsd-src/sys/net/if.c:1286 >> panic: witness_warn >> cpuid =3D 2 >> time =3D 1773824038 >> KDB: stack backtrace: >> db_trace_self_wrapper() at db_trace_self_wrapper+0xa5/frame = 0xfffffe00ec001710 >> kdb_backtrace() at kdb_backtrace+0xc6/frame 0xfffffe00ec001870 >> vpanic() at vpanic+0x214/frame 0xfffffe00ec001a10 >> panic() at panic+0xb5/frame 0xfffffe00ec001ad0 >> witness_warn() at witness_warn+0x7f7/frame 0xfffffe00ec001c30 >> userret() at userret+0x11c/frame 0xfffffe00ec001d10 >> amd64_syscall() at amd64_syscall+0x694/frame 0xfffffe00ec001f30 >> fast_syscall_common() at fast_syscall_common+0xf8/frame = 0xfffffe00ec001f30 >> --- syscall (19, FreeBSD ELF64, compat.lseek), rip =3D 0xfd288de65ba, = rsp =3D 0xfd284aad4b8, rbp =3D 0xfd284aad500 --- >> Uptime: 29m8s >> Dumping 1355 out of 16137 = MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% >> ``` >>=20 >> The line 1286 of sys/net/if.c is ( my WIP revision ), >>=20 >> ``` >> 1250 static int >> 1251 if_vmove_reclaim(struct thread *td, char *ifname, int jid) >> 1252 { >> ... >> 1282 /* Get interface back from child jail/vnet. */ >> 1283 found =3D if_unlink_ifnet(ifp, true); >> 1284 MPASS(found); >> 1285 sx_xlock(&ifnet_detach_sxlock); >> 1286 if_vmove(ifp, vnet_dst); >> 1287 sx_xunlock(&ifnet_detach_sxlock); >> 1288=20 >> 1289 /* Report the new if_xname back to the userland. */ >> 1290 sprintf(ifname, "%s", ifp->if_xname); >> 1291=20 >> 1292 prison_free(pr); >> 1293 CURVNET_RESTORE(); >> 1294 return (0); >> 1295 } >> ``` >>=20 >> That puzzled me a lot. The current thread only reaches to = `if_vmove()` , how does it happen to userret() ? >>=20 >> Is that witness warn is false report ? >=20 > I doubt that this is false report. >=20 > userret() is the common code that prepares thread for return from = kernel > to usermode. As part of the preparation, it checks that no locks is > left owned by the thread. What you see is the result of the failing > check. >=20 > In other words, you have some syscall that locked a lock, did not = released > it, then returned. The layer that manages return to userspace caught = the > issue. Thanks ! I'll take care of the locking issue ( syscall returned without = lock released ). I'll report back if I re-encounter this issue. Best regards, Zhenlei --Apple-Mail=_17AB9DFF-2CE9-4D2B-8EC7-0CC734F90566 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii

On Mar 19, 2026, at 2:01 AM, Konstantin Belousov <kostikbel@gmail.com>= wrote:

On Wed, Mar 18, 2026 at 05:36:00PM +0800, Zhenlei Huang = wrote:
Hi,

While I'm working on the if_detach() / = if_vmove() racing issue and I wrote
a test and try to = repeat the race condition and got this assert panic,

```
<6>epair1b: link state changed to = UP
userret: returning with the following locks held:
exclusive sx ifnet_detach_sx (ifnet_detach_sx) r =3D 1 = (0xffffffff83ab75e0) locked @ = /home/zlei/freebsd-src/sys/net/if.c:1286
panic: = witness_warn
cpuid =3D 2
time =3D = 1773824038
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0xa5/frame = 0xfffffe00ec001710
kdb_backtrace() at = kdb_backtrace+0xc6/frame 0xfffffe00ec001870
vpanic() at = vpanic+0x214/frame 0xfffffe00ec001a10
panic() at = panic+0xb5/frame 0xfffffe00ec001ad0
witness_warn() at = witness_warn+0x7f7/frame 0xfffffe00ec001c30
userret() at = userret+0x11c/frame 0xfffffe00ec001d10
amd64_syscall() at = amd64_syscall+0x694/frame 0xfffffe00ec001f30
fast_syscall_common() at fast_syscall_common+0xf8/frame = 0xfffffe00ec001f30
--- syscall (19, FreeBSD ELF64, = compat.lseek), rip =3D 0xfd288de65ba, rsp =3D 0xfd284aad4b8, rbp =3D = 0xfd284aad500 ---
Uptime: 29m8s
Dumping 1355 = out of 16137 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91%
```

The line 1286 of = sys/net/if.c is ( my WIP revision ),

```
1250 static int
1251 if_vmove_reclaim(struct = thread *td, char *ifname, int jid)
1252 {
...
1282 =         /* Get interface back = from child jail/vnet. */
1283 =         found =3D = if_unlink_ifnet(ifp, true);
1284 =         MPASS(found);
1285 =         sx_xlock(&ifnet_detach= _sxlock);
1286 =         if_vmove(ifp, = vnet_dst);
1287 =         sx_xunlock(&ifnet_deta= ch_sxlock);
1288 
1289 =         /* Report the new = if_xname back to the userland. */
1290 =         sprintf(ifname, "%s", = ifp->if_xname);
1291 
1292 =         prison_free(pr);
1293 =         CURVNET_RESTORE();
1294         return = (0);
1295 }
```

That puzzled me a lot. The current thread only reaches to = `if_vmove()` , how does it happen to userret() ?

Is that witness warn is false report ?

I doubt that this is false report.

userret() is = the common code that prepares thread for return from kernel
to usermode. =  As part of the preparation, it checks that  no locks = is
left owned by = the thread.  What you see is the result of the failing
check.

In other words, you have some syscall that locked a lock, did = not released
it, then = returned.  The layer that manages return to userspace caught = the
issue.

Thanks ! I'll take care of the locking issue = (  syscall returned without lock = released ).

I'll report back if I = re-encounter this issue.

Best regards,
Zhenlei

= --Apple-Mail=_17AB9DFF-2CE9-4D2B-8EC7-0CC734F90566--