From nobody Mon Aug 17 12:10:21 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hNs9968vFz6pBx4 for ; Mon, 17 Aug 2026 12:10:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hNs995Kgzz3DxL for ; Mon, 17 Aug 2026 12:10:21 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786968621; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Yj3SlJv1F5PQ/EH813ygjbHf37JYwbWkHI2c7hriO3o=; b=qA7EpT4hiYE0XguLdJZ+Vryqqa6RncfQYvrvyoXaB6cR2dh/IHEmVY5nQEi03PfcnQySB+ Tc0dOUSUyvWtLzqZHKF2SQ8K3uKOdfFktvbuT7sWP5373UkNULAFTniIh3/eTBW8HHk24U ug120wa49SxxKCA0k6JtUtOB2b7TLRLkC8cBV3gDQS1xygW0Ks8u2YZAuP5ByUYBIsnnXo 1WgNT1wrJvEd6RQvwgkKf2ikIiV62mItZ77/8PTmV3eT/Vqw2GJowjd/oI0fC433CF1kh3 CXqijPK+2ZqKwDf4db1zfIQ+JbDEgy7ouU+bPA+sMelLRtFkp4zkQxExFxwElQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1786968621; a=rsa-sha256; cv=none; b=OZO5SKWW4CZ2it7pEs49btcjeGnUXyDUOw1QZtKGCfXDRzr3tw5tpx2EfF7r/pMXHFff20 IMkW15PZkQeFcdOSluSVhbf5FeLOmfEt0hZut4rF4cLzE/+7OAbKdEgfVPrOTsGgeGD2/G /M0qXTikTyf/KWc9MOTw4STA3ZlwjohAj55B4lFwn19CozU/mQP0yrTMj2107XO2+Qplx3 GwSPoEZo/W82u9dEMIlC1SAF5mBgO44S11lrNNGkCpm/4H7wSB6t2Xjo3rvcryOB1/9NR1 VcFvj44Dj1jzxJJiYIKFWBMewA2K8fgKGA1KHStIAxMsxg57qY7WH91kYAydjw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786968621; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Yj3SlJv1F5PQ/EH813ygjbHf37JYwbWkHI2c7hriO3o=; b=vYxCm1VaWYAC6TP37+JLCHIS/9Bl06rbhpJ3feTDx4WaJXIQ5jSvRVa8BzksyH6vqew6pN T677vdmGaoaMYoh0TDqxAGu8e3fNi7nYDeoRbkFWMXjByGjKicuchAym9C5iQTSYvfmgg/ MjwtB+VCRF/IsE5ZpABdVEjmQS3l+F/IKnPILjX925uUDxIFhIzFwddDmTBd0jotoFamMK gJ8ZLBzJo+I7TH/RGYQaBN86/oOTTCTu8aeRxm2plx8NVGDgX6u9XO+GP8yyAAA4YEfIuN gpQ2Uz4nkEP2IDVXyZ6+csFV31SPSKO0pOwKM9wfrwoNn8ZPsU+35A9xyTd4vg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hNs993zjdz1CZv for ; Mon, 17 Aug 2026 12:10:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 3a76c by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Mon, 17 Aug 2026 12:10:21 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org Cc: Sujithra Periasamy From: Mark Johnston Subject: git: 873ba78dfe6d - stable/15 - gve: Implement AQ batching for queue creation and destruction List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: 873ba78dfe6d644d1de124a096f46aaae5a9599e Auto-Submitted: auto-generated Date: Mon, 17 Aug 2026 12:10:21 +0000 Message-Id: <6a82fa2d.3a76c.109c95de@gitrepo.freebsd.org> The branch stable/15 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=873ba78dfe6d644d1de124a096f46aaae5a9599e commit 873ba78dfe6d644d1de124a096f46aaae5a9599e Author: Sujithra Periasamy AuthorDate: 2026-08-07 13:07:20 +0000 Commit: Mark Johnston CommitDate: 2026-08-17 12:07:41 +0000 gve: Implement AQ batching for queue creation and destruction Currently, the FreeBSD driver configures and destroys queues sequentially by issuing individual Admin Queue (AQ) commands. During queue teardown (e.g., interface reset), disabling queues one by one leaves the device in a partially configured state. Because the device does not yet know that the driver is in the process of fully unconfiguring all queues, this intermediate state can trigger transient error logs (such as when queue 0 is disabled while other queues are still active). Modify the driver to use Admin Queue batching for both the creation and destruction of TX and RX queues. Commands are now queued and kicked together, ensuring the queue configuration changes are applied atomically and preventing transient errors from being logged. Signed-off-by: Sujithra Periasamy Reviewed by: markj MFC after: 1 week Sponsored by: Google Differential Revision: https://reviews.freebsd.org/D58696 (cherry picked from commit 36d57489ca07642dec31390e90c25b2a0ca9313e) --- sys/dev/gve/gve_adminq.c | 47 +++++++++++++++++++++++++++++++++++++---------- 1 file changed, 37 insertions(+), 10 deletions(-) diff --git a/sys/dev/gve/gve_adminq.c b/sys/dev/gve/gve_adminq.c index 9b59570a2af4..43952308098b 100644 --- a/sys/dev/gve/gve_adminq.c +++ b/sys/dev/gve/gve_adminq.c @@ -214,6 +214,9 @@ gve_process_device_options(struct gve_priv *priv, return (0); } +static int gve_adminq_issue_cmd(struct gve_priv *priv, + struct gve_adminq_command *cmd); +static int gve_adminq_kick_and_wait(struct gve_priv *priv); static int gve_adminq_execute_cmd(struct gve_priv *priv, struct gve_adminq_command *cmd); @@ -225,7 +228,7 @@ gve_adminq_destroy_tx_queue(struct gve_priv *priv, uint32_t id) cmd.opcode = htobe32(GVE_ADMINQ_DESTROY_TX_QUEUE); cmd.destroy_tx_queue.queue_id = htobe32(id); - return (gve_adminq_execute_cmd(priv, &cmd)); + return (gve_adminq_issue_cmd(priv, &cmd)); } static int @@ -236,7 +239,7 @@ gve_adminq_destroy_rx_queue(struct gve_priv *priv, uint32_t id) cmd.opcode = htobe32(GVE_ADMINQ_DESTROY_RX_QUEUE); cmd.destroy_rx_queue.queue_id = htobe32(id); - return (gve_adminq_execute_cmd(priv, &cmd)); + return (gve_adminq_issue_cmd(priv, &cmd)); } int @@ -248,13 +251,18 @@ gve_adminq_destroy_rx_queues(struct gve_priv *priv, uint32_t num_queues) for (i = 0; i < num_queues; i++) { err = gve_adminq_destroy_rx_queue(priv, i); if (err != 0) { - device_printf(priv->dev, "Failed to destroy rxq %d, err: %d\n", + device_printf(priv->dev, "Failed to issue destroy rxq %d, err: %d\n", i, err); + return (err); } } - if (err != 0) + err = gve_adminq_kick_and_wait(priv); + if (err != 0) { + device_printf(priv->dev, "Failed to batch destroy rx queues, err: %d\n", + err); return (err); + } device_printf(priv->dev, "Destroyed %d rx queues\n", num_queues); return (0); @@ -269,13 +277,18 @@ gve_adminq_destroy_tx_queues(struct gve_priv *priv, uint32_t num_queues) for (i = 0; i < num_queues; i++) { err = gve_adminq_destroy_tx_queue(priv, i); if (err != 0) { - device_printf(priv->dev, "Failed to destroy txq %d, err: %d\n", + device_printf(priv->dev, "Failed to issue destroy txq %d, err: %d\n", i, err); + return (err); } } - if (err != 0) + err = gve_adminq_kick_and_wait(priv); + if (err != 0) { + device_printf(priv->dev, "Failed to batch destroy tx queues, err: %d\n", + err); return (err); + } device_printf(priv->dev, "Destroyed %d tx queues\n", num_queues); return (0); @@ -325,7 +338,7 @@ gve_adminq_create_rx_queue(struct gve_priv *priv, uint32_t queue_index) htobe16(priv->rx_buf_size_dqo); } - return (gve_adminq_execute_cmd(priv, &cmd)); + return (gve_adminq_issue_cmd(priv, &cmd)); } int @@ -337,12 +350,19 @@ gve_adminq_create_rx_queues(struct gve_priv *priv, uint32_t num_queues) for (i = 0; i < num_queues; i++) { err = gve_adminq_create_rx_queue(priv, i); if (err != 0) { - device_printf(priv->dev, "Failed to create rxq %d, err: %d\n", + device_printf(priv->dev, "Failed to issue create rxq %d, err: %d\n", i, err); goto abort; } } + err = gve_adminq_kick_and_wait(priv); + if (err != 0) { + device_printf(priv->dev, "Failed to batch create rx queues, err: %d\n", + err); + goto abort; + } + if (bootverbose) device_printf(priv->dev, "Created %d rx queues\n", num_queues); return (0); @@ -381,7 +401,7 @@ gve_adminq_create_tx_queue(struct gve_priv *priv, uint32_t queue_index) cmd.create_tx_queue.tx_comp_ring_size = htobe16(priv->tx_desc_cnt); } - return (gve_adminq_execute_cmd(priv, &cmd)); + return (gve_adminq_issue_cmd(priv, &cmd)); } int @@ -393,12 +413,19 @@ gve_adminq_create_tx_queues(struct gve_priv *priv, uint32_t num_queues) for (i = 0; i < num_queues; i++) { err = gve_adminq_create_tx_queue(priv, i); if (err != 0) { - device_printf(priv->dev, "Failed to create txq %d, err: %d\n", + device_printf(priv->dev, "Failed to issue create txq %d, err: %d\n", i, err); goto abort; } } + err = gve_adminq_kick_and_wait(priv); + if (err != 0) { + device_printf(priv->dev, "Failed to batch create tx queues, err: %d\n", + err); + goto abort; + } + if (bootverbose) device_printf(priv->dev, "Created %d tx queues\n", num_queues); return (0);