From nobody Sun Apr 30 06:58:15 2023 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q8HHD0BdGz48sBg; Sun, 30 Apr 2023 06:58:16 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q8HHC6Wnbz3P18; Sun, 30 Apr 2023 06:58:15 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1682837895; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=6ugO0Nju/m9CmbxYsywlnr95Has3nD2Kl6h2z1aHCdQ=; b=ilIJFIU6Jw+T4SLnPwUJ4Hbrwyny3SopciuFr+Xib/0Xg01vMh+b1CvtTndd72ScEd8cz8 U9OdsE02CDtUhvKl5HAc/orR40wVnyWsL9BurNo9tNqpTIViu+56M5jSGFyXQMvRWwqUDO P8ZhfDSz9WLrZCQN4G9zeDDF2nxgKp5bSNSgeLpedqTBnGtjXqrG9QACfO4DYOMMXgBt3q 8mC5IE/1D1popKJe1F5sWNX0luumaGWUppese3fuYrgDKK0ZhhrHZP5Ll56Sw3inKdjNA1 ThuqTvue99bJHVoKb/hR4yk284kRuDkMZppqh5tudIHsr+Uipp5W15fresXGzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1682837895; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=6ugO0Nju/m9CmbxYsywlnr95Has3nD2Kl6h2z1aHCdQ=; b=vaDRfwC9hD/Py38vAeLtFpFQQyA9kXO44cIFFT+kVxz2H2FmFpMQZs4eZozS7RyMqjFc4f UE8Cu5Lyfekq8LWDyW5QBf1as25H/FPQAQeFtxWFt1RTTMAqkK9/v97WbeaIkKAfnDSzZc Sg1LAx3A6HYCadjU6IhdMnsVCx1xRbAhJ/g1VR61Gk7721UtkiqOXwWl7HUU9vTV2ZPWMA np72Zw62eo43G82O/NN8HraDfh10ppoE/wSDjGEtIVuEjiqY9UxeXnGpWtcxUgum9OKfWk L8UUta1h9111LZtVatYWVLL73YpF+P/Su2sgMWBsFXVQN60I3u+jz0IPAGM/qg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1682837895; a=rsa-sha256; cv=none; b=puIKSqa0S9g0UQ6Wunpu0GIpDHxSGpoI3xjvBlmBY/QFQlJEB65PUPYS/q+HDPndj5n9mT /yo9HrVWmCeQioSFK98xaL66S0+sKhWekl4j/R+eGCZGIPCofPmxeFH9UZAkI4HK8BEWmM 3Mq5dqkBz028TPN4DOgk5RXEqddzSjL9kbLitcqSR99D0LtfRvXaC7071Hea4/N2fl/4D5 V11n4dNNGHvtl0hgJjNtecHInWMoufgI4kXKS8EyUni/UfcrIyYnT6t17b/YoSRv/sqPHi sMfqciAMq1U7rfpTlM6YRFiJwnWs8XzYdA0HvoDrK7WNtW246KiWrXQVdCjZHg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Q8HHC5cslzDrB; Sun, 30 Apr 2023 06:58:15 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 33U6wFPA071959; Sun, 30 Apr 2023 06:58:15 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 33U6wFSn071958; Sun, 30 Apr 2023 06:58:15 GMT (envelope-from git) Date: Sun, 30 Apr 2023 06:58:15 GMT Message-Id: <202304300658.33U6wFSn071958@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Hans Petter Selasky Subject: git: 3d052f97cef5 - stable/13 - mlx5: Make MLX5_COMP_EQ_SIZE tunable. List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: hselasky X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 3d052f97cef570f0b65b8c93936e690af6d37acb Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by hselasky: URL: https://cgit.FreeBSD.org/src/commit/?id=3d052f97cef570f0b65b8c93936e690af6d37acb commit 3d052f97cef570f0b65b8c93936e690af6d37acb Author: Hans Petter Selasky AuthorDate: 2023-04-18 11:11:02 +0000 Commit: Hans Petter Selasky CommitDate: 2023-04-30 06:56:19 +0000 mlx5: Make MLX5_COMP_EQ_SIZE tunable. When using hardware pacing, this value can be increased, because more SQ's means more EQ events aswell. Make it tunable, hw.mlx5.comp_eq_size . Sponsored by: NVIDIA Networking (cherry picked from commit 3bb3e4768ff854b88ba0a7d129edad49f15d7ce3) --- sys/dev/mlx5/driver.h | 4 ---- sys/dev/mlx5/mlx5_core/mlx5_main.c | 23 ++++++++++++++++++++++- 2 files changed, 22 insertions(+), 5 deletions(-) diff --git a/sys/dev/mlx5/driver.h b/sys/dev/mlx5/driver.h index e1766ae8a742..cdf8deb34a6c 100644 --- a/sys/dev/mlx5/driver.h +++ b/sys/dev/mlx5/driver.h @@ -831,10 +831,6 @@ struct mlx5_core_dct { u16 uid; }; -enum { - MLX5_COMP_EQ_SIZE = 1024, -}; - enum { MLX5_PTYS_IB = 1 << 0, MLX5_PTYS_EN = 1 << 2, diff --git a/sys/dev/mlx5/mlx5_core/mlx5_main.c b/sys/dev/mlx5/mlx5_core/mlx5_main.c index 431a277119cf..9862b9ecec56 100644 --- a/sys/dev/mlx5/mlx5_core/mlx5_main.c +++ b/sys/dev/mlx5/mlx5_core/mlx5_main.c @@ -86,6 +86,11 @@ SYSCTL_INT(_hw_mlx5, OID_AUTO, fast_unload_enabled, CTLFLAG_RWTUN, &mlx5_fast_unload_enabled, 0, "Set to enable fast unload. Clear to disable."); +static int mlx5_core_comp_eq_size = 1024; +SYSCTL_INT(_hw_mlx5, OID_AUTO, comp_eq_size, CTLFLAG_RDTUN | CTLFLAG_MPSAFE, + &mlx5_core_comp_eq_size, 0, + "Set default completion EQ size between 1024 and 16384 inclusivly. Value should be power of two."); + static LIST_HEAD(intf_list); static LIST_HEAD(dev_list); static DEFINE_MUTEX(intf_mutex); @@ -180,6 +185,22 @@ static struct mlx5_profile profiles[] = { }, }; +static int +mlx5_core_get_comp_eq_size(void) +{ + int value = mlx5_core_comp_eq_size; + + if (value < 1024) + value = 1024; + else if (value > 16384) + value = 16384; + + /* make value power of two, rounded down */ + while (value & (value - 1)) + value &= (value - 1); + return (value); +} + static void mlx5_set_driver_version(struct mlx5_core_dev *dev) { const size_t driver_ver_sz = @@ -688,7 +709,7 @@ static int alloc_comp_eqs(struct mlx5_core_dev *dev) INIT_LIST_HEAD(&table->comp_eqs_list); ncomp_vec = table->num_comp_vectors; - nent = MLX5_COMP_EQ_SIZE; + nent = mlx5_core_get_comp_eq_size(); for (i = 0; i < ncomp_vec; i++) { eq = kzalloc_node(sizeof(*eq), GFP_KERNEL, dev->priv.numa_node);