From nobody Mon Oct 27 14:15:09 2025 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4cwFrs5Yqqz6DjQL; Mon, 27 Oct 2025 14:15:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R12" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4cwFrs53pZz3cbw; Mon, 27 Oct 2025 14:15:09 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1761574509; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=uuMO7RK/bYzQeZhv/QbWe+Lw3lJYQxEgSHHygbCaFMw=; b=pmyT8LMVDE1eWezC6dNRsiUpncZ18xNaDExST+r7V3SNVkuL4aHqXokcV0EAxLRuZfP48z QqD/W5lRJ59aFvCd6LxCZqhkG6azHXIYmjKl7IdDun+pn7RMAqDzfoS8l28eBNezkipOE1 oNl99++iRsge93BD6BulAWZYQq+6PfHBZ5ztGUl6teceyoq9NvLjIyRSsWikfcTMxMI4vG xpljQpZoSk6RSK8ZK/wvCIcnL89FkvYvFrdqseRo0A+4dLU0ILYh1r6hGtUmBDn5K80DkN sww9Twf0KdBlK77WVdu7Nv41VR5R3bhLKMDRhtmbJ+FgedxOJycR8UkILFsMJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1761574509; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=uuMO7RK/bYzQeZhv/QbWe+Lw3lJYQxEgSHHygbCaFMw=; b=LZ1qMyebHQ20a3aOTI62YNo2RWOd3ao9L5DTr0G0gdqlWKCsoQQOHWs4vEzeajlF8DvywA JnC7loQ8gQeszYDA6BqKOGkSPUmcUf9dQkPzw8Xcwpr5AkcTOasOvZSuug92FJcile+796 F3zWTzKvHyTb/cM2sb3mrR+JqrtVFCbQL/xbGCqVaLZ9A7dOqnMugm+KroSdhVoAcyr3wM gcux5+VT2oDFhp0UStCfJvOUSNDJdHdWpKa6gLgnLc5ad4a4xzBLoS9e/2VeJuY+PFnKu1 DY1J+vNAeR5xJ9BM52w9we4L7Gdd9Ac/yryCGGxQivIliBmJ07ba5WdswinWIQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1761574509; a=rsa-sha256; cv=none; b=TRKXy0yAbvcfZNJpsGx1tlOoUmJhzWn7L98gAlA0jhcL2nVrhKyvI+WJOE1JIfQPk1Lndc PKLgTYbUhXtg4yN2KzZX1ZlAaOHXI3CDJ1sjQDu0R6vwT/QRck24P72yXAwCnZEUApm/hh eTwN/vtYJ9rejC9IZqCchPjXla/5Hig4M1OoRa3H+EmK0jHEGIt4LOfTcb7h+mK+RHcOQ2 lT1mHGXG2aDNas7OOFgU3x2Z614uy77bgE9qXYazGFNrrs2/bMInqHNENh4o3EEVMi2lQH et9Yf28DfcBznSnq0SCOyptJrbabSFSw27a9+1+MMaWCVTV/58TyDNqbb+0QjQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4cwFrs4cT1z1LNw; Mon, 27 Oct 2025 14:15:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 59REF9xw024900; Mon, 27 Oct 2025 14:15:09 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 59REF9kg024897; Mon, 27 Oct 2025 14:15:09 GMT (envelope-from git) Date: Mon, 27 Oct 2025 14:15:09 GMT Message-Id: <202510271415.59REF9kg024897@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Andrew Gallatin Subject: git: 23b46b2bbf0a - main - audit: convert audit event class lookup to lockless List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: gallatin X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 23b46b2bbf0a8f3a740b6e5baf0767816afc339e Auto-Submitted: auto-generated The branch main has been updated by gallatin: URL: https://cgit.FreeBSD.org/src/commit/?id=23b46b2bbf0a8f3a740b6e5baf0767816afc339e commit 23b46b2bbf0a8f3a740b6e5baf0767816afc339e Author: Andrew Gallatin AuthorDate: 2025-10-27 14:05:49 +0000 Commit: Andrew Gallatin CommitDate: 2025-10-27 14:15:01 +0000 audit: convert audit event class lookup to lockless When system call auditing is enabled, every audited call does a lookup in the evclass hash table. This table appears to be insert only (eg, nothing can be removed) and protecting it with an rwlock is overkill. Using an rwlock causes just the atomic operations to maintain uncontended rwlock state to be responsible for measurable overhead on high core count servers making lots of system calls. Given that the evclass hash table can never have items removed, only added, using a mutex to serialize additions and converting to ck_list allows sufficient protection for lockless lookups. In a contrived example of 64 cores, all reading 1 byte from their own file, this change increases performance from 5M reads/sec to 70M reads/sec on an AMD 7502P. Reviewed by: markj, mjg, glebius (privately) Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D53176 --- sys/security/audit/audit_bsm_db.c | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/sys/security/audit/audit_bsm_db.c b/sys/security/audit/audit_bsm_db.c index c9f3d5c8a549..358162544287 100644 --- a/sys/security/audit/audit_bsm_db.c +++ b/sys/security/audit/audit_bsm_db.c @@ -56,6 +56,8 @@ #include #include +#include + /* * Hash table functions for the audit event number to event class mask * mapping. @@ -64,21 +66,21 @@ struct evclass_elem { au_event_t event; au_class_t class; - LIST_ENTRY(evclass_elem) entry; + CK_LIST_ENTRY(evclass_elem) entry; }; struct evclass_list { - LIST_HEAD(, evclass_elem) head; + CK_LIST_HEAD(, evclass_elem) head; }; static MALLOC_DEFINE(M_AUDITEVCLASS, "audit_evclass", "Audit event class"); -static struct rwlock evclass_lock; static struct evclass_list evclass_hash[EVCLASSMAP_HASH_TABLE_SIZE]; - -#define EVCLASS_LOCK_INIT() rw_init(&evclass_lock, "evclass_lock") -#define EVCLASS_RLOCK() rw_rlock(&evclass_lock) -#define EVCLASS_RUNLOCK() rw_runlock(&evclass_lock) -#define EVCLASS_WLOCK() rw_wlock(&evclass_lock) -#define EVCLASS_WUNLOCK() rw_wunlock(&evclass_lock) +static struct mtx evclass_mtx; +#define EVCLASS_LOCK_INIT() mtx_init(&evclass_mtx, "evclass_lock", NULL, MTX_DEF) +#define EVCLASS_WLOCK() mtx_lock(&evclass_mtx); +#define EVCLASS_WUNLOCK() mtx_unlock(&evclass_mtx); +/* make these do something if we ever remove entries from the hash */ +#define EVCLASS_RLOCK() {} +#define EVCLASS_RUNLOCK() {} /* * Hash table maintaining a mapping from audit event numbers to audit event @@ -118,7 +120,7 @@ au_event_class(au_event_t event) EVCLASS_RLOCK(); evcl = &evclass_hash[event % EVCLASSMAP_HASH_TABLE_SIZE]; class = 0; - LIST_FOREACH(evc, &evcl->head, entry) { + CK_LIST_FOREACH(evc, &evcl->head, entry) { if (evc->event == event) { class = evc->class; goto out; @@ -150,7 +152,7 @@ au_evclassmap_insert(au_event_t event, au_class_t class) EVCLASS_WLOCK(); evcl = &evclass_hash[event % EVCLASSMAP_HASH_TABLE_SIZE]; - LIST_FOREACH(evc, &evcl->head, entry) { + CK_LIST_FOREACH(evc, &evcl->head, entry) { if (evc->event == event) { evc->class = class; EVCLASS_WUNLOCK(); @@ -161,7 +163,7 @@ au_evclassmap_insert(au_event_t event, au_class_t class) evc = evc_new; evc->event = event; evc->class = class; - LIST_INSERT_HEAD(&evcl->head, evc, entry); + CK_LIST_INSERT_HEAD(&evcl->head, evc, entry); EVCLASS_WUNLOCK(); } @@ -172,7 +174,7 @@ au_evclassmap_init(void) EVCLASS_LOCK_INIT(); for (i = 0; i < EVCLASSMAP_HASH_TABLE_SIZE; i++) - LIST_INIT(&evclass_hash[i].head); + CK_LIST_INIT(&evclass_hash[i].head); /* * Set up the initial event to class mapping for system calls.