From nobody Mon Mar 31 19:00:12 2025 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZRL6h71kQz5sQdZ; Mon, 31 Mar 2025 19:00:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZRL6h6HBfz3Bnx; Mon, 31 Mar 2025 19:00:12 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1743447612; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=I3f+pyWisSaOwDmGczje9/ufXV75Dv+FKBE8+JYrVO8=; b=nNrfEanti2qs50qNhJMDrzeDUKvM/aPuc+rzQSebq5JuZ2qTrj1dPRbS9M8H2p894AMGTK 2RSz0g+gqLiql+rYbGgRSlhnny+MftwuH6hHazKf64yC2FNmVbtF5G9KSuRBe0I1kPdysQ FHn1PLGaSb2yAN1lC5QvxRPVHitowhfobzo9BXJ8K6zcNWBmoY97UzdiWtc79a8apCDj/1 vlYSAFufL8fP92HeUwq4CaEWvsB68XmacWFOnYOpgZIof14wfqbodlM19mOYGCK6/pDoeK aHQoH/l5bitKRB64M3e3imgnNP+YL8Urd+5zVvZelK9xs3R1Q/SlNtffIuM4bA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1743447612; a=rsa-sha256; cv=none; b=vzwB3lWrTW5mDJdXxb0vYJ2uRZD5eNuf2DIhMymYXT0omqthVM1jyr8lssGK0PfDE/l09V XZaYj7OxGY6l8BdrAJUSKxmxhvDU1FQofIAKBPpMxVijSTU6ajAnF7oUk1MbaE+/6bvDmX B+hVMXK1rLUN30KA3MWL6CWngMhInG0Qkh/atZqLv9Uusufw8fk9mHsqNlolutMrURe3HM 4ys0BkciluW5XcRlCeMC1qZdElzSvFwBGzLczdQQOBGYfgjYtJer0NATssnzFdADeD927k 7QnaDRFzape+k4AtoXF4XKP4WeRFxYKs8PAG+xVLMGSGfZYew39V8aByG4V7cA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1743447612; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=I3f+pyWisSaOwDmGczje9/ufXV75Dv+FKBE8+JYrVO8=; b=xMogji7skazlr8vEDAynfWSuN5qAifC+lY31D7UGsoZCrDvYCP460EIQfEjpxbwW/qC1qH 7XwNR5j5Y8zDipPiGVX5EKPZpIdLSfoJmzL4ZgrJyuRrzs+ubYTtpQViO7lsEHl5BTT8Dy l2ofGVG43zvlaMokHVCQgIEgc8+XgXHO+CjiY2xMYysru0I469rPfNb/swYk3u30r+ob/E xV10GzctQO4zv11GgevhZxDOfpV8YfNcvJaqJYsC2UguwE/anzavCsDQHgJ8qLh02FSXzE MXwu28cAb96OKcW1RoK2XrDJo+I2lzhin1awt1LvusU5w5n+NJXXN+efm6fGeA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4ZRL6h572LzxF6; Mon, 31 Mar 2025 19:00:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 52VJ0CFX001193; Mon, 31 Mar 2025 19:00:12 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 52VJ0CmD001184; Mon, 31 Mar 2025 19:00:12 GMT (envelope-from git) Date: Mon, 31 Mar 2025 19:00:12 GMT Message-Id: <202503311900.52VJ0CmD001184@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Konstantin Belousov Subject: git: f0adc907fc7d - main - mlx5en: sync channel close with the rq completion processing List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: f0adc907fc7d3eebfc692fd5f4987c97e61b103d Auto-Submitted: auto-generated The branch main has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=f0adc907fc7d3eebfc692fd5f4987c97e61b103d commit f0adc907fc7d3eebfc692fd5f4987c97e61b103d Author: Konstantin Belousov AuthorDate: 2025-03-04 07:45:17 +0000 Commit: Konstantin Belousov CommitDate: 2025-03-31 18:59:50 +0000 mlx5en: sync channel close with the rq completion processing Without the wait, mlx5e_destroy_rq() might free mbuf that is passed up to the network stack on receive in mlx5e_poll_rx_cq(). Sponsored by: NVidia networking MFC after: 1 week --- sys/dev/mlx5/mlx5_en/en.h | 1 + sys/dev/mlx5/mlx5_en/mlx5_en_main.c | 11 +++++++++++ sys/dev/mlx5/mlx5_en/mlx5_en_rx.c | 5 +++++ 3 files changed, 17 insertions(+) diff --git a/sys/dev/mlx5/mlx5_en/en.h b/sys/dev/mlx5/mlx5_en/en.h index fdc12381e91b..f59902be226a 100644 --- a/sys/dev/mlx5/mlx5_en/en.h +++ b/sys/dev/mlx5/mlx5_en/en.h @@ -772,6 +772,7 @@ struct mlx5e_rq { struct mlx5e_cq cq; struct lro_ctrl lro; volatile int enabled; + int processing; int ix; /* Dynamic Interrupt Moderation */ diff --git a/sys/dev/mlx5/mlx5_en/mlx5_en_main.c b/sys/dev/mlx5/mlx5_en/mlx5_en_main.c index 2c83492bdaa9..c3539cf8b8f2 100644 --- a/sys/dev/mlx5/mlx5_en/mlx5_en_main.c +++ b/sys/dev/mlx5/mlx5_en/mlx5_en_main.c @@ -1470,6 +1470,17 @@ static void mlx5e_close_rq_wait(struct mlx5e_rq *rq) { + mtx_lock(&rq->mtx); + MPASS(rq->enabled == 0); + while (rq->processing > 0) { + /* + * No wakeup, relying on timeout. + * Use msleep_sbt() since msleep() conflicts with linuxkpi. + */ + msleep_sbt(&rq->processing, &rq->mtx, 0, "mlx5ecrq", + tick_sbt * hz, 0, C_HARDCLOCK); + } + mtx_unlock(&rq->mtx); mlx5e_disable_rq(rq); mlx5e_close_cq(&rq->cq); cancel_work_sync(&rq->dim.work); diff --git a/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c b/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c index 3aef10109720..6b53db6fea23 100644 --- a/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c +++ b/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c @@ -697,6 +697,9 @@ mlx5e_rx_cq_comp(struct mlx5_core_cq *mcq, struct mlx5_eqe *eqe __unused) mtx_unlock(&c->iq.lock); mtx_lock(&rq->mtx); + if (rq->enabled == 0) + goto out; + rq->processing++; /* * Polling the entire CQ without posting new WQEs results in @@ -717,6 +720,8 @@ mlx5e_rx_cq_comp(struct mlx5_core_cq *mcq, struct mlx5_eqe *eqe __unused) net_dim(&rq->dim, rq->stats.packets, rq->stats.bytes); mlx5e_cq_arm(&rq->cq, MLX5_GET_DOORBELL_LOCK(&rq->channel->priv->doorbell_lock)); tcp_lro_flush_all(&rq->lro); + rq->processing--; +out: mtx_unlock(&rq->mtx); for (int j = 0; j != MLX5E_MAX_TX_NUM_TC; j++) {