From nobody Mon Sep 02 09:12:46 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wy31p6xqYz5MrBd; Mon, 02 Sep 2024 09:12:46 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wy31p4xlgz4C9r; Mon, 2 Sep 2024 09:12:46 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1725268366; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=f6XGelX87Mj17vnyDmJKjUYg3G2afxC5k6pPsF8aKew=; b=Oe2CrfTiTsMDbBaZ2s/GsJvauVXm12Tm0l68rcKGVfuDnBz4oQIrEcJhphpsoVmOOIlm8i SYJRCxfvWa3Ih7A+6fiG53NmeIYaOkkR7C9wBJqzkMD9/iEEG3ueDhdE7/OA8WpbNdiSWc wXGSJcI3dQ1WM0sK/dOh3UdWFjd40AKX7HM5XCdu26C4mQdMXXhN12p0cULJ4mV8onHXCq OEtIC+NfvbgrFKC6Gjfscfa5/t5lIUcqnU+Ln+lIyPlbycMhzRpQFW+0j/+UdiTcc4q8Io fITAilzuHg9w4HIj/M1PB84X+DS5yr0xleQFxOEFHXb3UHFdDyn+Z1f8bSXQ5A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1725268366; a=rsa-sha256; cv=none; b=QtRhp4nMgHPpoUOG8b45oaXD1ncFcU+ZjdrJKr3eBwdK7yCnLiwqcDz8jihPeQST/Gjqd+ t4mxe8YyiAyLPMwV7MbWTwCse8u23irtBgiq/uLrJYKIgdxJie9QHn17r8FevL+k5SxMqU oCxpkSK83I+RDA/T41B5AfizSpDmsTyBpHRRIsdS4ipBKvYM4ORJ9B6f+cyJd18yqnC7pb 8Vu8IKb63N7Jo2wyJ1YHJlytS/QZ6klDhLiDjksdXdUlUZWns4dNMSoxb6SmASNCzQyR+c nR+Sw26XBTTYMYr2D24MQvouwaZlQEEPKH/BtNYk6KYAivEPaYbaHRc6300h0g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1725268366; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=f6XGelX87Mj17vnyDmJKjUYg3G2afxC5k6pPsF8aKew=; b=cNG9bZnkBPjYlZ9hfyk+T/NKcETv/YtTVuT+D4sdv7d5t/YE29m0oWQ+NRmgS34/ReJRmB FL1KvmH4YXUKa3Tdymh27NeDaqFr2mBiLEm/qjnBYP0kv/aVeFkN7VvyETGA27lRUV0ca4 xC7lWw6TMbAoVkKd8CaixntRwvJ+TylT3XBYjbT3rjM2NNMIXGXsqLOW9Fv/3VhF9ajcj6 A3gwWSt+ngMj7V8OxsBaxEl2HuKwiCJZg1ktxaLDcPp0eGx4m65erfAj0yJWhLxl6AWYxy ruoNX/Zl3tNN8N8SoPurnzzpP9vGVF5+PFLaxHot6AweSVWntC4P4yKeeF6epA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Wy31p4YlJz16Dn; Mon, 2 Sep 2024 09:12:46 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 4829Ckmo049933; Mon, 2 Sep 2024 09:12:46 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 4829CkXS049930; Mon, 2 Sep 2024 09:12:46 GMT (envelope-from git) Date: Mon, 2 Sep 2024 09:12:46 GMT Message-Id: <202409020912.4829CkXS049930@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Andrew Turner Subject: git: 3c15d02bb8a1 - stable/13 - buf_ring: Use atomic operations with br_cons_tail List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 3c15d02bb8a193d7c38cb9689454e4f26d12bb21 Auto-Submitted: auto-generated The branch stable/13 has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=3c15d02bb8a193d7c38cb9689454e4f26d12bb21 commit 3c15d02bb8a193d7c38cb9689454e4f26d12bb21 Author: Andrew Turner AuthorDate: 2024-08-19 09:06:52 +0000 Commit: Andrew Turner CommitDate: 2024-09-02 09:11:57 +0000 buf_ring: Use atomic operations with br_cons_tail Use an atomic operation with a memory barrier loading br_cons_tail from the producer thread and storing to it in the consumer thread. On dequeue we need to read the pointer value from the buf_ring before moving the consumer tail as that indicates the entry is available to be used. The store release atomic operation guarantees this. In the enqueueing thread we then need to use a load acquire atomic operation to ensure writing to this entry can only happen after the tail has been read and checked. Reported by: Ali Saidi Co-developed by: Ali Saidi Reviewed by: markj Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D46152 (cherry picked from commit 44e1cfca417c5ef0db908f3836ec3ba704ef1de2) --- sys/sys/buf_ring.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/sys/sys/buf_ring.h b/sys/sys/buf_ring.h index dec0f971ae44..9aeb5aa19fa8 100644 --- a/sys/sys/buf_ring.h +++ b/sys/sys/buf_ring.h @@ -91,7 +91,7 @@ buf_ring_enqueue(struct buf_ring *br, void *buf) do { prod_head = br->br_prod_head; prod_next = prod_head + 1; - cons_tail = br->br_cons_tail; + cons_tail = atomic_load_acq_32(&br->br_cons_tail); if ((int32_t)(cons_tail + br->br_prod_size - prod_next) < 1) { rmb(); @@ -229,7 +229,7 @@ buf_ring_dequeue_sc(struct buf_ring *br) panic("inconsistent list cons_tail=%d cons_head=%d", br->br_cons_tail, cons_head); #endif - br->br_cons_tail = cons_next; + atomic_store_rel_32(&br->br_cons_tail, cons_next); return (buf); } @@ -257,7 +257,7 @@ buf_ring_advance_sc(struct buf_ring *br) #ifdef DEBUG_BUFRING br->br_ring[cons_head & mask] = NULL; #endif - br->br_cons_tail = cons_next; + atomic_store_rel_32(&br->br_cons_tail, cons_next); } /*