From nobody Thu Mar 13 16:03:13 2025 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ZDC2p2yj5z5qs2R; Thu, 13 Mar 2025 16:03:14 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ZDC2n70WLz3QZv; Thu, 13 Mar 2025 16:03:13 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1741881794; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Ac13CZgAiSKIZARxXdqdQ+CyyO7nAFbrpkYS0RLofDw=; b=UxPpPs+SmEIo2V+5dmUtmWXCLp0V+2jlVbkM3Mhh5m45EQ1CNuKXk7PRX8idl9i3QYCAlM yCv8BkZjtY9J8hI6HEv+JkndAaTWIEjD1YzZKOOUhR6sOUB7Kp3cHZ9FIBkuz2MVQz2SwA yrhkXjlyZsoZH3IDUOnti561K1QKhXRVcNOhC0l2QYBFlTRk7pEkjh7nUjyNG0opX/kG/3 UzqFt/5EgX0T2iW0Qs30tMF+krdt5+kroQ2mc+uQghKTAggiJZukeFPgOS7IcTh1EWbY9o jdA5NV0difZyKcB/sAXFg9k2zGqKRTcV2BCLqHZ/oKFJ5Y2KoUEiFao2s1WIrQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1741881794; a=rsa-sha256; cv=none; b=Ofzw0qlxdwlLPMoG+yF2IAXMESjZyi3UCxpgUc/IdkqLpwQeZUWd3l+lURJyR77rtqYqPL Zott1pEBH+MVIuANNIDYHNPnid/21IFZ6pj6lGHDun0EDPtvCq1I06FQ5DIPqgz2v/4QuS wLr330SgvQ/j1BOlZh9S1qAgDk7p7ihjYMOCPQ3CGODKfn15VUn1Ta4PSa3wyvFzZlk95/ eDc4ZLQlnVHwRVP3YBLTzXPrZm/vIK1ywZOy5e9wZ+/8/jNNfNvaDqsK1vLhhzeBaDim9g DjGOWTYAurNY9/2G32Zcc/ZS6RDQHyMnrMr0/P/PIzLgisu3qY3qtqdoLLm/LQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1741881794; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Ac13CZgAiSKIZARxXdqdQ+CyyO7nAFbrpkYS0RLofDw=; b=P/y4B3DFZ211gcNXS0xu/adGeXaPEe6UOsujm7ErrnbWMeaEfSR5wy5ThhcdoU987rznev +W3wZ+CXVaVMAE3lmFDzR5fjJ5dHOgPLonNQqYX1xc8cpHR1mNmhm5Y9KARbNgrdoJzUiR Il8cEvT2/ik/dQZfZLg+D7XcJn3FLOjMfkmRswZ5PZMLc5Baz6P2xocFa1l5nnNzzcIKI/ dCJpzHHLgKfWzRlyT0ze0aIeuKseM4Ik+WQPiWaGJP/Ka0gUcY9B/zTG3Igl6NY6hmJKwE UkC3CGsrcd0oWtX1l0Ka4WgA8A1Fifmjq9UlbQW0S04QtDmEBUsTXvDS8hDFcg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4ZDC2n5pPKz14tJ; Thu, 13 Mar 2025 16:03:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 52DG3DBl048534; Thu, 13 Mar 2025 16:03:13 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 52DG3DmR048531; Thu, 13 Mar 2025 16:03:13 GMT (envelope-from git) Date: Thu, 13 Mar 2025 16:03:13 GMT Message-Id: <202503131603.52DG3DmR048531@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Konstantin Belousov Subject: git: 02fe38b92175 - main - mlx5en: make the hw lro control dynamic List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 02fe38b92175cb8f3a77f7a2bb72afb83836ebd2 Auto-Submitted: auto-generated The branch main has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=02fe38b92175cb8f3a77f7a2bb72afb83836ebd2 commit 02fe38b92175cb8f3a77f7a2bb72afb83836ebd2 Author: Konstantin Belousov AuthorDate: 2025-01-29 15:45:18 +0000 Commit: Konstantin Belousov CommitDate: 2025-03-13 15:58:53 +0000 mlx5en: make the hw lro control dynamic Reviewed by: Ariel Ehrenberg , Slava Shwartsman Sponsored by: NVidia networking MFC after: 1 week --- sys/dev/mlx5/mlx5_en/en.h | 1 + sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c | 3 ++ sys/dev/mlx5/mlx5_en/mlx5_en_main.c | 83 ++++++++++++++++++++++++++++------ 3 files changed, 73 insertions(+), 14 deletions(-) diff --git a/sys/dev/mlx5/mlx5_en/en.h b/sys/dev/mlx5/mlx5_en/en.h index 80e0b7fbdedb..bcc33824a5f5 100644 --- a/sys/dev/mlx5/mlx5_en/en.h +++ b/sys/dev/mlx5/mlx5_en/en.h @@ -1301,6 +1301,7 @@ void mlx5e_refresh_sq_inline(struct mlx5e_priv *priv); int mlx5e_update_buf_lossy(struct mlx5e_priv *priv); int mlx5e_fec_update(struct mlx5e_priv *priv); int mlx5e_hw_temperature_update(struct mlx5e_priv *priv); +int mlx5e_hw_lro_update_tirs(struct mlx5e_priv *priv); /* Internal Queue, IQ, API functions */ void mlx5e_iq_send_nop(struct mlx5e_iq *, u32); diff --git a/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c b/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c index 459801cdf27d..b0b7a175a1b6 100644 --- a/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c +++ b/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c @@ -1133,6 +1133,9 @@ mlx5e_ethtool_handler(SYSCTL_HANDLER_ARGS) priv->params.hw_lro_en = false; priv->params_ethtool.hw_lro = 0; } + + error = mlx5e_hw_lro_update_tirs(priv); + /* restart network interface, if any */ if (was_opened) mlx5e_open_locked(priv->ifp); diff --git a/sys/dev/mlx5/mlx5_en/mlx5_en_main.c b/sys/dev/mlx5/mlx5_en/mlx5_en_main.c index 67583afc21c7..321ed8ac9976 100644 --- a/sys/dev/mlx5/mlx5_en/mlx5_en_main.c +++ b/sys/dev/mlx5/mlx5_en/mlx5_en_main.c @@ -2944,6 +2944,73 @@ mlx5e_get_rss_key(void *key_ptr) #endif } +static void +mlx5e_hw_lro_set_tir_ctx_lro_max_msg_sz(struct mlx5e_priv *priv, u32 *tirc) +{ +#define ROUGH_MAX_L2_L3_HDR_SZ 256 + + MLX5_SET(tirc, tirc, lro_max_msg_sz, (priv->params.lro_wqe_sz - + ROUGH_MAX_L2_L3_HDR_SZ) >> 8); +} + +static void +mlx5e_hw_lro_set_tir_ctx(struct mlx5e_priv *priv, u32 *tirc) +{ + MLX5_SET(tirc, tirc, lro_enable_mask, + MLX5_TIRC_LRO_ENABLE_MASK_IPV4_LRO | + MLX5_TIRC_LRO_ENABLE_MASK_IPV6_LRO); + /* TODO: add the option to choose timer value dynamically */ + MLX5_SET(tirc, tirc, lro_timeout_period_usecs, + MLX5_CAP_ETH(priv->mdev, lro_timer_supported_periods[2])); + mlx5e_hw_lro_set_tir_ctx_lro_max_msg_sz(priv, tirc); +} + +static int +mlx5e_hw_lro_update_tir(struct mlx5e_priv *priv, int tt, bool inner_vxlan) +{ + struct mlx5_core_dev *mdev = priv->mdev; + u32 *in; + void *tirc; + int inlen; + int err; + + inlen = MLX5_ST_SZ_BYTES(modify_tir_in); + in = mlx5_vzalloc(inlen); + if (in == NULL) + return (-ENOMEM); + tirc = MLX5_ADDR_OF(modify_tir_in, in, tir_context); + + /* fill the command part */ + MLX5_SET(modify_tir_in, in, tirn, inner_vxlan ? + priv->tirn_inner_vxlan[tt] : priv->tirn[tt]); + MLX5_SET64(modify_tir_in, in, modify_bitmask, + (1 << MLX5_MODIFY_TIR_BITMASK_LRO)); + + /* fill the context */ + if (priv->params.hw_lro_en) + mlx5e_hw_lro_set_tir_ctx(priv, tirc); + + err = mlx5_core_modify_tir(mdev, in, inlen); + + kvfree(in); + return (err); +} + +int +mlx5e_hw_lro_update_tirs(struct mlx5e_priv *priv) +{ + int err, err1, i; + + err = 0; + for (i = 0; i != 2 * MLX5E_NUM_TT; i++) { + err1 = mlx5e_hw_lro_update_tir(priv, i / 2, (i % 2) ? true : + false); + if (err1 != 0 && err == 0) + err = err1; + } + return (-err); +} + static void mlx5e_build_tir_ctx(struct mlx5e_priv *priv, u32 * tirc, int tt, bool inner_vxlan) { @@ -2954,8 +3021,6 @@ mlx5e_build_tir_ctx(struct mlx5e_priv *priv, u32 * tirc, int tt, bool inner_vxla MLX5_SET(tirc, tirc, transport_domain, priv->tdn); -#define ROUGH_MAX_L2_L3_HDR_SZ 256 - #define MLX5_HASH_IP (MLX5_HASH_FIELD_SEL_SRC_IP |\ MLX5_HASH_FIELD_SEL_DST_IP) @@ -2968,18 +3033,8 @@ mlx5e_build_tir_ctx(struct mlx5e_priv *priv, u32 * tirc, int tt, bool inner_vxla MLX5_HASH_FIELD_SEL_DST_IP |\ MLX5_HASH_FIELD_SEL_IPSEC_SPI) - if (priv->params.hw_lro_en) { - MLX5_SET(tirc, tirc, lro_enable_mask, - MLX5_TIRC_LRO_ENABLE_MASK_IPV4_LRO | - MLX5_TIRC_LRO_ENABLE_MASK_IPV6_LRO); - MLX5_SET(tirc, tirc, lro_max_msg_sz, - (priv->params.lro_wqe_sz - - ROUGH_MAX_L2_L3_HDR_SZ) >> 8); - /* TODO: add the option to choose timer value dynamically */ - MLX5_SET(tirc, tirc, lro_timeout_period_usecs, - MLX5_CAP_ETH(priv->mdev, - lro_timer_supported_periods[2])); - } + if (priv->params.hw_lro_en) + mlx5e_hw_lro_set_tir_ctx(priv, tirc); if (inner_vxlan) MLX5_SET(tirc, tirc, tunneled_offload_en, 1);