From nobody Wed Nov 04 23:41:39 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jKhmM4wgLz70LpX for ; Wed, 04 Nov 2026 23:41:39 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jKhmM4Bb8z3M6p for ; Wed, 04 Nov 2026 23:41:39 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793835699; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=sPtL9RyIIVsgItsQu8gMcv+283zCt+47fwzO+QEsVdc=; b=W4DgX/swSowDTm+HSoY4MobqroKqH0cr2i+JR5izS0vozR/2wT40uUG1Uyz8h8r7tz03Ht Q7gYqyQo7D+eKrg61tVXeNTB4IrGyeF3R+p2p8a0iGzFp9CXLXXSI1e57U6QJ8oo5oI8fB DrLHAsv/0MJdCP8DmcrP+yd6YJfRTSKBxD3y6aSrW9z4MJoIwzWmn7rCe0ad9mJswui3Ty npl8CkQAP8oYSzVJUR+sTl+ENtxK4ofexp2hBXQRJ9VndufMroLziDnU7edh/PHIVOdGyU lf376Lk6EeZPCwqKRBa34HKhkNacebOGc/i++VWg8lfbGHTwSVFf1k6vdWTn+g== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1793835699; b=dwhjavKPYvfW+HIq4I8HnnPv+f8ae38o8zNoG8decupOnGdtcuRlTzRhMW6cj3Au6xGQvh ohtI+xpudMWcnhI+BggwxwFSY6v/n8pJAer0Pn5P6p75ETrjuTehJmMmsc49anhmVtYifQ 8qumfI9VpUO9t4ludEBkAI7hV/3SqMI7hMtdMveHU9gfXrCDGjlfbrujoKAvLaDnfGcSw4 hG7lGC8pPTpZm0OzY9JwMzVe6Q8hAefp8IebV6HH/WMhGUdrJU7t+B/r8f1oMe7a0GhIJ8 q11IQc1EHcyGMjz1CJfvTdufNeZpGGDojSZh62FKuPEvRLUmwc8bDIPVabNa4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793835699; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=sPtL9RyIIVsgItsQu8gMcv+283zCt+47fwzO+QEsVdc=; b=cwQI59SLWt9CgMLjdW0kiOO8Buv67vfJuwtBzoFqV9CVfoV4Ct0eBf0x27Xj6v/pHckd0m esfWXeB7oTYxMQn4yI3lGKH6ONL0l0sjEJJnJP4Vxcl2PXBLrnoCCSqRIy8lZMTddGimxr tgybBmCOw2ifPStihwIzGEMfQSD64cCpxkZ/DM/yQE27OSPvnz0WvSmHV0pe4jjgDVHJ7N oL862T1usaGd72QDWajz+uavTfl6xiGGyOvoKcUr6+PQk0FXRHC11HYNgoyNsV2i/nIv1/ M6xmzUHwjJDxprDI5EVYJKRZwX/2s9JhM3owH4KWhyjHTqmyybBNbyqRP9rLdQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jKhmM3sRkzP2w for ; Wed, 04 Nov 2026 23:41:39 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 38207 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Wed, 04 Nov 2026 23:41:39 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: ShengYi Hung Subject: git: 7806832f23fb - main - libthr: Cache recent pshared_key List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: Commit messages for all branches of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: aokblast X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 7806832f23fb31d28386d42692b221776c9f4ee4 Auto-Submitted: auto-generated Date: Wed, 04 Nov 2026 23:41:39 +0000 Message-Id: <6aebc2b3.38207.57306871@gitrepo.freebsd.org> The branch main has been updated by aokblast: URL: https://cgit.FreeBSD.org/src/commit/?id=7806832f23fb31d28386d42692b221776c9f4ee4 commit 7806832f23fb31d28386d42692b221776c9f4ee4 Author: ShengYi Hung AuthorDate: 2026-10-21 07:03:27 +0000 Commit: ShengYi Hung CommitDate: 2026-11-04 23:41:05 +0000 libthr: Cache recent pshared_key For a pshare key, we can actually cache the last one as a fast path instead of trapping into kernel directly. It makes sense as a lock and unlock should be a pair on the same key and at the same time. For example, lock(A) -> unlock(A) -> lock(B) -> unlock(B). Each unlock is a hit and refresh later (50% hit). For a single pshare lock, it is evne a 100% hit as the key never changed. For nested lock lock(A) -> lock(B) -> lock(C) -> unlock(C) -> unlock(B) -> unlock(A). It only cache the last one. However, we should really think about rewritting this logic as it is a pshare lock shared by many processes and nested should be uncommon. Reviewed by: kib MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D60957 --- lib/libthr/thread/thr_private.h | 4 ++++ lib/libthr/thread/thr_pshared.c | 45 +++++++++++++++++++++++++++++++++++++++-- 2 files changed, 47 insertions(+), 2 deletions(-) diff --git a/lib/libthr/thread/thr_private.h b/lib/libthr/thread/thr_private.h index ffc4ff7233db..4469533bc40e 100644 --- a/lib/libthr/thread/thr_private.h +++ b/lib/libthr/thread/thr_private.h @@ -522,6 +522,10 @@ struct pthread { /* Number rwlocks rdlocks held. */ int rdlock_count; + void *pshared_key; + void *pshared_val; + unsigned long pshared_gen; + /* * Current locks bitmap for rtld. */ int rtld_bits; diff --git a/lib/libthr/thread/thr_pshared.c b/lib/libthr/thread/thr_pshared.c index 698cc5e6adc7..bebae21601a6 100644 --- a/lib/libthr/thread/thr_pshared.c +++ b/lib/libthr/thread/thr_pshared.c @@ -47,9 +47,14 @@ LIST_HEAD(pshared_hash_head, psh); static struct pshared_hash_head pshared_hash[HASH_SIZE]; #define PSHARED_KEY_HASH(key) (((unsigned long)(key) >> 8) % HASH_SIZE) /* XXXKIB: lock could be split to per-hash chain, if appears contested */ -static struct urwlock pshared_lock = DEFAULT_URWLOCK; +static struct urwlock pshared_lock __aligned(CACHE_LINE_SIZE) = DEFAULT_URWLOCK; static int page_size; +static struct { + unsigned long val; + char pad[CACHE_LINE_SIZE - sizeof(unsigned long)]; +} pshared_gen __aligned(CACHE_LINE_SIZE); + #define GC_INTERVAL_USEC 25000 static struct timeval pshared_gc_time = {0, 0}; bool __thr_pshared_destroy_imm_gc = false; @@ -93,6 +98,30 @@ pshared_unlock(struct pthread *curthread) _thr_ast(curthread); } +static void +pshared_gen_bump(void) +{ + atomic_add_long(&pshared_gen.val, 1); +} + +static void * +pshared_cache_lookup(struct pthread *curthread, void *key) +{ + if (curthread->pshared_key == key && + curthread->pshared_gen == atomic_load_long(&pshared_gen.val)) + return (curthread->pshared_val); + return (NULL); +} + +static void +pshared_cache_insert(struct pthread *curthread, void *key, void *val, + unsigned long gen) +{ + curthread->pshared_key = key; + curthread->pshared_val = val; + curthread->pshared_gen = gen; +} + /* * Among all processes sharing a lock only one executes * pthread_lock_destroy(). Other processes still have the hash and @@ -128,6 +157,7 @@ pshared_gc(struct pthread *curthread) if (error == 0) continue; LIST_REMOVE(h, link); + pshared_gen_bump(); munmap(h->val, page_size); free(h); } @@ -212,6 +242,7 @@ pshared_remove(void *key) LIST_FOREACH(h, hd, link) { if (h->key == key) { LIST_REMOVE(h, link); + pshared_gen_bump(); val = h->val; free(h); return (val); @@ -245,6 +276,7 @@ __thr_pshared_offpage(void *key, int doalloc) { struct pthread *curthread; void *res; + unsigned long gen; int fd, ins_done; curthread = _get_curthread(); @@ -252,11 +284,17 @@ __thr_pshared_offpage(void *key, int doalloc) pshared_destroy(curthread, key); res = NULL; } else { + res = pshared_cache_lookup(curthread, key); + if (res != NULL) + return (res); pshared_rlock(curthread); + gen = pshared_gen.val; res = pshared_lookup(key); pshared_unlock(curthread); - if (res != NULL) + if (res != NULL) { + pshared_cache_insert(curthread, key, res, gen); return (res); + } } fd = _umtx_op(NULL, UMTX_OP_SHM, doalloc ? UMTX_SHM_CREAT : UMTX_SHM_LOOKUP, key, NULL); @@ -267,11 +305,14 @@ __thr_pshared_offpage(void *key, int doalloc) if (res == MAP_FAILED) return (NULL); pshared_wlock(curthread); + gen = pshared_gen.val; ins_done = pshared_insert(key, &res); pshared_unlock(curthread); if (!ins_done) { pshared_clean(key, res); res = NULL; + } else { + pshared_cache_insert(curthread, key, res, gen); } return (res); }