From nobody Tue Feb 01 15:24:35 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id D325B1916CF8; Tue, 1 Feb 2022 15:24:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jp7yX15bjz3G1n; Tue, 1 Feb 2022 15:24:36 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1643729076; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=z7stjpFGNhSfBZ0djb7NIre/RZqZWXP9D6/SbShDg6o=; b=OiutuqIWOZnno9hLYEXKh21n8YYTQLkMITonODbynHv6B7A7BtNCCwmXeOBMqH7MZj/Vui JvGuYsTfpYnawbXBC8kSkHLVaEcuTq0+NdaAfx0SorvXaYgQrvR/Ucrg0gk90I0Jcx+LIF 5mjDoSRDshVw3/k8o/xN/Kz81U3bY4OuVJNMB5HQght38OlZ/lgrmI94NW4Q+rIQP1Lfb7 1DAUKnKbl1GxAbNb5nv0nLfmXb1/G4Qst6cp9KqpuoeeExqrL3K41TgvLXklu2obbd+9tQ xtmtwq2WGPiXYHrYJaEC8ssp7D3mVPBF+Gj6IPAbmqGB7IADzuDyVhdWcRvzDA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 01F1E13608; Tue, 1 Feb 2022 15:24:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 211FOZX5051346; Tue, 1 Feb 2022 15:24:35 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 211FOZpM051345; Tue, 1 Feb 2022 15:24:35 GMT (envelope-from git) Date: Tue, 1 Feb 2022 15:24:35 GMT Message-Id: <202202011524.211FOZpM051345@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Hans Petter Selasky Subject: git: ea00d7e8cab9 - main - mlx5: Add raw ethernet local loopback support. List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: hselasky X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: ea00d7e8cab97f818c9793a112249ab79a2f8322 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1643729076; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=z7stjpFGNhSfBZ0djb7NIre/RZqZWXP9D6/SbShDg6o=; b=DClfCb2EtILx3+DqIE/rqaOCLfBhkTjhMul0rsRlrBQLyIpdKjtshU8PYpP5KzX3nMrk1t DaHswN2TdwEo6Uv4rKfUJqTp7F2Ix8jdOzaSET4OeGmzeNGwH/b/EXvyZ/ybe6qJ7ndqBs ddBhJ/vBez9lDnsDjveoxNDoSTG6Udi0DxxrGJox39sA/DEUe3uIMEGrL5/bsrXLiJOKl4 adDh+HOVoghYeNrZiDAd8Y+Pz1oUbMdYZj84IgA6Ql7wvQuam0LV8WBDnFS8LLk5miUDZZ XkblKEMNCnKDi3fojKNb82dueQNSx81ttkAX6GaY4n+AojYxL8fpfaKEkHTvrg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1643729076; a=rsa-sha256; cv=none; b=K+YliTGr8YEEpKEvniSPTLrECavcm+jr0LfZQ3mUBjECNVLHxcQBukecZceFwtGD17ybp0 j/VtBXHbNqoKE9vIdRKYJ1nFH+JtI9viQg9aghBa9L2nWugxqEgy6DaLuJ5alujd7uQQGr LFpkF0yKTSr3xF6tyuVU7rNRcs2mSUQpLLYuLuoPv1dkiWAcq6g41KdIWpaZsIlATKpHLj q5e+yNZ8zg4J1VvbkzcOdvp0KvxDnxxmHz4jGfNjc+ESgUCjzYVOg2Ve+NnPxHprkobB7L wvsp/AzBLiHJJDpcJByD6pbdOuWH+kZ3zHPt2e+kTw9CgXbNhe/hBFwFY1CAqA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by hselasky: URL: https://cgit.FreeBSD.org/src/commit/?id=ea00d7e8cab97f818c9793a112249ab79a2f8322 commit ea00d7e8cab97f818c9793a112249ab79a2f8322 Author: Hans Petter Selasky AuthorDate: 2022-02-01 15:20:15 +0000 Commit: Hans Petter Selasky CommitDate: 2022-02-01 15:21:16 +0000 mlx5: Add raw ethernet local loopback support. Currently, unicast/multicast loopback raw ethernet (non-RDMA) packets are sent back to the vport. A unicast loopback packet is the packet with destination MAC address the same as the source MAC address. For multicast, the destination MAC address is in the vport's multicast filter list. Moreover, the local loopback is not needed if there is one or none user space context. After this patch, the raw ethernet unicast and multicast local loopback are disabled by default. When there is more than one user space context, the local loopback is enabled. Note that when local loopback is disabled, raw ethernet packets are not looped back to the vport and are forwarded to the next routing level (eswitch, or multihost switch, or out to the wire depending on the configuration). Linux commits: c85023e153e3824661d07307138fdeff41f6d86a 8978cc921fc7fad3f4d6f91f1da01352aeeeff25 MFC after: 1 week Sponsored by: NVIDIA Networking --- sys/dev/mlx5/mlx5_core/mlx5_main.c | 17 +++++++++++++++++ sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c | 12 ++++++++++++ sys/dev/mlx5/mlx5_ib/mlx5_ib.h | 4 ++++ sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c | 32 +++++++++++++++++++++++++++++++- 4 files changed, 64 insertions(+), 1 deletion(-) diff --git a/sys/dev/mlx5/mlx5_core/mlx5_main.c b/sys/dev/mlx5/mlx5_core/mlx5_main.c index 0745d19b2725..aaa8f657432a 100644 --- a/sys/dev/mlx5/mlx5_core/mlx5_main.c +++ b/sys/dev/mlx5/mlx5_core/mlx5_main.c @@ -532,6 +532,17 @@ static int set_hca_ctrl(struct mlx5_core_dev *dev) return err; } +static int mlx5_core_set_hca_defaults(struct mlx5_core_dev *dev) +{ + int ret = 0; + + /* Disable local_lb by default */ + if (MLX5_CAP_GEN(dev, port_type) == MLX5_CAP_PORT_TYPE_ETH) + ret = mlx5_nic_vport_update_local_lb(dev, false); + + return ret; +} + static int mlx5_core_enable_hca(struct mlx5_core_dev *dev, u16 func_id) { u32 out[MLX5_ST_SZ_DW(enable_hca_out)] = {0}; @@ -1135,6 +1146,12 @@ static int mlx5_load_one(struct mlx5_core_dev *dev, struct mlx5_priv *priv, goto err_free_comp_eqs; } + err = mlx5_core_set_hca_defaults(dev); + if (err) { + mlx5_core_err(dev, "Failed to set HCA defaults %d\n", err); + goto err_free_comp_eqs; + } + err = mlx5_mpfs_init(dev); if (err) { mlx5_core_err(dev, "mpfs init failed %d\n", err); diff --git a/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c b/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c index d07aee1f0793..f57d70080a46 100644 --- a/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c +++ b/sys/dev/mlx5/mlx5_en/mlx5_en_ethtool.c @@ -1230,6 +1230,12 @@ mlx5e_ethtool_handler(SYSCTL_HANDLER_ARGS) break; case MLX5_PARAM_OFFSET(mc_local_lb): + /* check if mlx5ib is managing this feature */ + if (MLX5_CAP_GEN(priv->mdev, port_type) != MLX5_CAP_PORT_TYPE_ETH) { + error = EOPNOTSUPP; + break; + } + priv->params_ethtool.mc_local_lb = priv->params_ethtool.mc_local_lb ? 1 : 0; @@ -1242,6 +1248,12 @@ mlx5e_ethtool_handler(SYSCTL_HANDLER_ARGS) break; case MLX5_PARAM_OFFSET(uc_local_lb): + /* check if mlx5ib is managing this feature */ + if (MLX5_CAP_GEN(priv->mdev, port_type) != MLX5_CAP_PORT_TYPE_ETH) { + error = EOPNOTSUPP; + break; + } + priv->params_ethtool.uc_local_lb = priv->params_ethtool.uc_local_lb ? 1 : 0; diff --git a/sys/dev/mlx5/mlx5_ib/mlx5_ib.h b/sys/dev/mlx5/mlx5_ib/mlx5_ib.h index ba4b49f24831..316a45806966 100644 --- a/sys/dev/mlx5/mlx5_ib/mlx5_ib.h +++ b/sys/dev/mlx5/mlx5_ib/mlx5_ib.h @@ -795,6 +795,10 @@ struct mlx5_ib_dev { struct mlx5_ib_congestion congestion; struct mlx5_async_ctx async_ctx; + + /* protect the user_td */ + struct mutex lb_mutex; + u32 user_td; }; static inline struct mlx5_ib_cq *to_mibcq(struct mlx5_core_cq *mcq) diff --git a/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c b/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c index 598a506b3b70..70a7fcd758de 100644 --- a/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c +++ b/sys/dev/mlx5/mlx5_ib/mlx5_ib_main.c @@ -1206,7 +1206,22 @@ static int mlx5_ib_alloc_transport_domain(struct mlx5_ib_dev *dev, u32 *tdn, if (err) return err; - return 0; + if ((MLX5_CAP_GEN(dev->mdev, port_type) != MLX5_CAP_PORT_TYPE_ETH) || + (!MLX5_CAP_GEN(dev->mdev, disable_local_lb_uc) && + !MLX5_CAP_GEN(dev->mdev, disable_local_lb_mc))) + return 0; + + mutex_lock(&dev->lb_mutex); + dev->user_td++; + + if (dev->user_td == 2) + err = mlx5_nic_vport_update_local_lb(dev->mdev, true); + + mutex_unlock(&dev->lb_mutex); + + if (err != 0) + mlx5_dealloc_transport_domain(dev->mdev, *tdn, uid); + return err; } static void mlx5_ib_dealloc_transport_domain(struct mlx5_ib_dev *dev, u32 tdn, @@ -1216,6 +1231,19 @@ static void mlx5_ib_dealloc_transport_domain(struct mlx5_ib_dev *dev, u32 tdn, return; mlx5_dealloc_transport_domain(dev->mdev, tdn, uid); + + if ((MLX5_CAP_GEN(dev->mdev, port_type) != MLX5_CAP_PORT_TYPE_ETH) || + (!MLX5_CAP_GEN(dev->mdev, disable_local_lb_uc) && + !MLX5_CAP_GEN(dev->mdev, disable_local_lb_mc))) + return; + + mutex_lock(&dev->lb_mutex); + dev->user_td--; + + if (dev->user_td < 2) + mlx5_nic_vport_update_local_lb(dev->mdev, false); + + mutex_unlock(&dev->lb_mutex); } static int mlx5_ib_alloc_ucontext(struct ib_ucontext *uctx, @@ -3284,6 +3312,8 @@ static void *mlx5_ib_add(struct mlx5_core_dev *mdev) MLX5_INIT_DOORBELL_LOCK(&dev->uar_lock); + mutex_init(&dev->lb_mutex); + INIT_IB_DEVICE_OPS(&dev->ib_dev.ops, mlx5, MLX5); snprintf(dev->ib_dev.name, IB_DEVICE_NAME_MAX, "mlx5_%d", device_get_unit(mdev->pdev->dev.bsddev)); dev->ib_dev.owner = THIS_MODULE;