From nobody Thu Oct 29 03:29:35 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jFV8c0KDJz6xfp6 for ; Thu, 29 Oct 2026 03:29:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jFV8b6jrHz4RTC for ; Thu, 29 Oct 2026 03:29:35 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793244575; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=jgWGwdGDyP5lJ78eC5lpV2S43+rOVQrHbZU/hPtGdX4=; b=S8unH3Sv7O1ZhtnIZYgk5G0dGGpLhpeu4oAW8TT2q6MM7hnKIZldNxp/8DYugG54Tyg9+p 2vTGUYlZ+uqtl3iaExGmWVGmmYTFPpDslEpLYlxTrKbWA2sJkvH7LMhh0ymzMblZk3R/CF DxOvhRO0X+mRrch10IfpRkGgzok7BQKQ2GUlgiH+Y7yTEZ2uZ9TUh/qSFnsZDrZ7aOqDG7 S5VuDM9sNmsvjcF2j7R0PJ/+X1aJi0J6tviUuEIcK+C7vWirtFES4HorGGE9W8o1du6V9w kwTtW7GxdypTc8iAMzBbMER/+wIdGEyJYCi5jKAmEwX4gzb4f5dt3aEOi2rgXg== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1793244575; b=HVj7aPM/1Alh8ErxGJmKbjeYacTPPFCS4CG+NOiGvo9b/1l0XwO0DZz9hgq8yADmIFqgEa TMNLoZbrrO+giNqHhCPDzrP9Py7JrLHrEoFlleOb0KHc8AeUFWkISUw7j6ooQfjASJ1LpP QIqkKaKP3unRwxAF/8FQv7Ev0ST6O5qEZEuS/3k+Y+SGV5H7ez9TWyG8/Puet2I5s2HGva TnnQWhoZ21LYsTarsUjSzOD2UCuUqMehdhesERySmp+eVHRk5VVax9pfjr03WCjFPUWtnZ u9ktBlPMzvu81wjIY4VXRmFSlKNhqHBJpokr5W7+iBiD9yCLxMZRdc6A6w2oLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793244575; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=jgWGwdGDyP5lJ78eC5lpV2S43+rOVQrHbZU/hPtGdX4=; b=WWLKJlQFyQzs2LhzwFnZ76l4n3BBsyTyfrst4fmTMKCbNqeGD0nm9rYY4nRkJCfizxnN2e /3V54F+fBPD+izUESeG1iasJ6MHv4KBMwmspSJgIUusVMfSpotNMfbe8wrYFGcBFCiG/jR XJ6y/Gy7fU7nq1gklBC1Bq9JuXbJ0YuLx2nAvmbaw5xNlaww3wa5Pkn6zC+/A31UzZmDzl rXvFthKlgPFHq0LJ5y5Byel/1YHO2xSmZUDGhMEXSXh3Yzqzl5yzGdqhArMTM/RokNUTWu l/X5mG0Ebk8PbaCUpDP/6uvRYNkhtRpEbc5GdwibJFnRKR4JY8aZbCbYhoVWPg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jFV8b6NLpzLjb for ; Thu, 29 Oct 2026 03:29:35 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 2179d by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Thu, 29 Oct 2026 03:29:35 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Cc: Gilad Ben Yakov From: Arthur Kiyanovski Subject: git: 3d1e732a43ed - main - ena: Retrieve RSS table size from device List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: Commit messages for all branches of the src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: akiyano X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 3d1e732a43eda16f20dea3fe2671ce6fca8a03a6 Auto-Submitted: auto-generated Date: Thu, 29 Oct 2026 03:29:35 +0000 Message-Id: <6ae2bd9f.2179d.51bb3be9@gitrepo.freebsd.org> The branch main has been updated by akiyano: URL: https://cgit.FreeBSD.org/src/commit/?id=3d1e732a43eda16f20dea3fe2671ce6fca8a03a6 commit 3d1e732a43eda16f20dea3fe2671ce6fca8a03a6 Author: Gilad Ben Yakov AuthorDate: 2026-03-30 12:12:50 +0000 Commit: Arthur Kiyanovski CommitDate: 2026-10-29 03:25:50 +0000 ena: Retrieve RSS table size from device The current driver assumes a default RSS table size of 128 entries, expecting the device's RSS table size to match this assumption. This patch allows the driver to create RSS indirection table with the size advertised by the device. Reviewed by: cperciva MFC after: 3 days Sponsored by: Amazon, Inc. --- share/man/man4/ena.4 | 1 - sys/dev/ena/ena.c | 5 ++++- sys/dev/ena/ena.h | 3 --- sys/dev/ena/ena_rss.c | 46 ++++++++++++++++++++++++++++++++++++++++------ sys/dev/ena/ena_rss.h | 35 +++++++++++++++++++++++++++++------ sys/dev/ena/ena_sysctl.c | 32 ++++++++++++++++++++++++-------- 6 files changed, 97 insertions(+), 25 deletions(-) diff --git a/share/man/man4/ena.4 b/share/man/man4/ena.4 index cb5b9367da43..453d30a5b64e 100644 --- a/share/man/man4/ena.4 +++ b/share/man/man4/ena.4 @@ -272,7 +272,6 @@ sysctl dev.ena.1.eni_metrics.sample_interval=10 .Ed .It Va dev.ena.X.rss.indir_table_size RSS indirection table size. -The default is 128. Returns the number of entries in the RSS indirection table. .Pp Example: diff --git a/sys/dev/ena/ena.c b/sys/dev/ena/ena.c index b489c58e81e5..211e33b8a4ce 100644 --- a/sys/dev/ena/ena.c +++ b/sys/dev/ena/ena.c @@ -4115,8 +4115,11 @@ ena_detach(device_t pdev) ena_free_pci_resources(adapter); - if (adapter->rss_indir != NULL) + if (adapter->rss_indir != NULL) { + free(adapter->rss_indir->table, M_DEVBUF); + free(adapter->rss_indir->sysctl_buf, M_DEVBUF); free(adapter->rss_indir, M_DEVBUF); + } if (likely(ENA_FLAG_ISSET(ENA_FLAG_RSS_ACTIVE, adapter))) ena_com_rss_destroy(ena_dev); diff --git a/sys/dev/ena/ena.h b/sys/dev/ena/ena.h index a62bfccba7ad..f54d7f115338 100644 --- a/sys/dev/ena/ena.h +++ b/sys/dev/ena/ena.h @@ -81,9 +81,6 @@ #define ENA_PKT_MAX_BUFS 19 -#define ENA_RX_RSS_TABLE_LOG_SIZE 7 -#define ENA_RX_RSS_TABLE_SIZE (1 << ENA_RX_RSS_TABLE_LOG_SIZE) - #define ENA_HASH_KEY_SIZE 40 #define ENA_MAX_FRAME_LEN 10000 diff --git a/sys/dev/ena/ena_rss.c b/sys/dev/ena/ena_rss.c index 88fb3c902f5d..4680e476bb19 100644 --- a/sys/dev/ena/ena_rss.c +++ b/sys/dev/ena/ena_rss.c @@ -102,15 +102,22 @@ ena_rss_init_default(struct ena_adapter *adapter) { struct ena_com_dev *ena_dev = adapter->ena_dev; device_t dev = adapter->pdev; + u32 table_size; int qid, rc, i; - rc = ena_com_rss_init(ena_dev, ENA_RX_RSS_TABLE_LOG_SIZE); + rc = ena_com_rss_init(ena_dev); if (unlikely(rc != 0)) { ena_log(dev, ERR, "Cannot init indirect table\n"); return (rc); } - for (i = 0; i < ENA_RX_RSS_TABLE_SIZE; i++) { + table_size = ena_get_rss_indirection_table_size(adapter); + if (table_size == 0) { + rc = EOPNOTSUPP; + goto err_rss_destroy; + } + + for (i = 0; i < table_size; i++) { #ifdef RSS qid = rss_get_indirection_to_bucket(i) % adapter->num_io_queues; #else @@ -227,6 +234,7 @@ SYSINIT(ena_rss_init, SI_SUB_KICK_SCHEDULER, SI_ORDER_SECOND, int ena_rss_indir_get(struct ena_adapter *adapter, uint32_t *table) { + u32 table_size; int rc, i; rc = ena_com_indirect_table_get(adapter->ena_dev, table); @@ -240,7 +248,11 @@ ena_rss_indir_get(struct ena_adapter *adapter, uint32_t *table) return (rc); } - for (i = 0; i < ENA_RX_RSS_TABLE_SIZE; ++i) + table_size = ena_get_rss_indirection_table_size(adapter); + if (table_size == 0) + return (EOPNOTSUPP); + + for (i = 0; i < table_size; ++i) table[i] = ENA_IO_RXQ_IDX_TO_COMBINED_IDX(table[i]); return (0); @@ -249,9 +261,14 @@ ena_rss_indir_get(struct ena_adapter *adapter, uint32_t *table) int ena_rss_indir_set(struct ena_adapter *adapter, uint32_t *table) { + u32 table_size; int rc, i; - for (i = 0; i < ENA_RX_RSS_TABLE_SIZE; ++i) { + table_size = ena_get_rss_indirection_table_size(adapter); + if (table_size == 0) + return (EOPNOTSUPP); + + for (i = 0; i < table_size; ++i) { rc = ena_com_indirect_table_fill_entry(adapter->ena_dev, i, ENA_IO_RXQ_IDX(table[i])); if (rc != 0) { @@ -275,21 +292,38 @@ int ena_rss_indir_init(struct ena_adapter *adapter) { struct ena_indir *indir = adapter->rss_indir; + u32 table_size; int rc; - if (indir == NULL) + if (indir == NULL) { + table_size = ena_get_rss_indirection_table_size(adapter); + if (table_size == 0) + return (EOPNOTSUPP); + adapter->rss_indir = indir = malloc(sizeof(struct ena_indir), M_DEVBUF, M_WAITOK | M_ZERO); + indir->table = malloc(sizeof(uint32_t) * table_size, + M_DEVBUF, M_WAITOK | M_ZERO); + indir->record_size = ENA_RSS_RECORD_SIZE(table_size, + adapter->max_num_io_queues); + indir->sysctl_buf = malloc(indir->record_size * table_size + 1, + M_DEVBUF, M_WAITOK | M_ZERO); + indir->table_size = table_size; + } + rc = ena_rss_indir_get(adapter, indir->table); if (rc != 0) { + free(indir->sysctl_buf, M_DEVBUF); + free(indir->table, M_DEVBUF); free(adapter->rss_indir, M_DEVBUF); adapter->rss_indir = NULL; return (rc); } - ena_rss_copy_indir_buf(indir->sysctl_buf, indir->table); + ena_rss_copy_indir_buf(indir->sysctl_buf, indir->table, + indir->table_size, indir->record_size); return (0); } diff --git a/sys/dev/ena/ena_rss.h b/sys/dev/ena/ena_rss.h index d1236ef26c33..3b6661b96e02 100644 --- a/sys/dev/ena/ena_rss.h +++ b/sys/dev/ena/ena_rss.h @@ -39,12 +39,21 @@ #include -#define ENA_RX_RSS_MSG_RECORD_SZ 8 +/* Number of decimal digits needed to represent a value in range [0, n-1] */ +#define ENA_DECIMAL_DIGITS(n) (snprintf(NULL, 0, "%d", n)) +/* + * Per-record size in sysctl_buf: " :\0" + * +1 for space/separator, +1 for ':', +1 for null terminator + */ +#define ENA_RSS_RECORD_SIZE(table_size, num_queues) \ + (1 + ENA_DECIMAL_DIGITS(table_size) + 1 + ENA_DECIMAL_DIGITS(num_queues) + 1) struct ena_indir { - uint32_t table[ENA_RX_RSS_TABLE_SIZE]; + uint32_t *table; /* This is the buffer wired to `rss.indir_table` sysctl. */ - char sysctl_buf[ENA_RX_RSS_TABLE_SIZE * ENA_RX_RSS_MSG_RECORD_SZ]; + char *sysctl_buf; + u32 table_size; + u32 record_size; }; int ena_rss_set_hash(struct ena_com_dev *ena_dev, const u8 *key); @@ -55,14 +64,28 @@ int ena_rss_indir_set(struct ena_adapter *adapter, uint32_t *table); int ena_rss_indir_init(struct ena_adapter *adapter); static inline void -ena_rss_copy_indir_buf(char *buf, uint32_t *table) +ena_rss_copy_indir_buf(char *buf, uint32_t *table, u32 table_size, + u32 record_size) { int i; - for (i = 0; i < ENA_RX_RSS_TABLE_SIZE; ++i) { - buf += snprintf(buf, ENA_RX_RSS_MSG_RECORD_SZ + 1, "%s%d:%d", + for (i = 0; i < table_size; ++i) { + buf += snprintf(buf, record_size, "%s%d:%d", i == 0 ? "" : " ", i, table[i]); } } +static inline u32 +ena_get_rss_indirection_table_size(struct ena_adapter *adapter) +{ + struct ena_com_dev *ena_dev = adapter->ena_dev; + + if (!ena_com_indirection_table_config_supported(ena_dev)) { + device_printf(adapter->pdev, + "RSS indirection table configuration not supported\n"); + return (0); + } + + return (1UL << ena_dev->rss.tbl_log_size); +} #endif /* !(ENA_RSS_H) */ diff --git a/sys/dev/ena/ena_sysctl.c b/sys/dev/ena/ena_sysctl.c index a46119e06adf..d244b015df41 100644 --- a/sys/dev/ena/ena_sysctl.c +++ b/sys/dev/ena/ena_sysctl.c @@ -54,6 +54,8 @@ static int ena_sysctl_metrics_interval(SYSCTL_HANDLER_ARGS); #ifndef RSS static int ena_sysctl_rss_key(SYSCTL_HANDLER_ARGS); static int ena_sysctl_rss_indir_table(SYSCTL_HANDLER_ARGS); +static int ena_sysctl_rss_indir_table_size(SYSCTL_HANDLER_ARGS); + #endif /* Limit max ENA sample rate to be an hour. */ @@ -160,8 +162,6 @@ SYSCTL_INT(_hw_ena, OID_AUTO, force_large_llq_header, CTLFLAG_RDTUN, &ena_force_large_llq_header, 0, "Change default LLQ entry size received from the device"); -int ena_rss_table_size = ENA_RX_RSS_TABLE_SIZE; - int ena_sysctl_allocate_customer_metrics_buffer(struct ena_adapter *adapter) { int rc = 0; @@ -632,9 +632,10 @@ ena_sysctl_add_rss(struct ena_adapter *adapter) ena_sysctl_rss_indir_table, "A", "RSS indirection table."); /* RSS indirection table size */ - SYSCTL_ADD_INT(ctx, child, OID_AUTO, "indir_table_size", - CTLFLAG_RD | CTLFLAG_MPSAFE, &ena_rss_table_size, 0, - "RSS indirection table size."); + SYSCTL_ADD_PROC(ctx, child, OID_AUTO, "indir_table_size", + CTLTYPE_INT | CTLFLAG_RD | CTLFLAG_MPSAFE, + adapter, 0, ena_sysctl_rss_indir_table_size, + "I", "RSS indirection table size."); } #endif /* RSS */ @@ -1166,7 +1167,8 @@ ena_sysctl_rss_indir_table(SYSCTL_HANDLER_ARGS) goto unlock; } - error = sysctl_handle_string(oidp, msg, sizeof(indir->sysctl_buf), req); + error = sysctl_handle_string(oidp, msg, indir->record_size + * indir->table_size, req); if (error != 0 || req->newptr == NULL) goto unlock; @@ -1186,7 +1188,7 @@ ena_sysctl_rss_indir_table(SYSCTL_HANDLER_ARGS) break; } - if (idx >= ENA_RX_RSS_TABLE_SIZE) { + if (idx >= indir->table_size) { device_printf(adapter->pdev, "Index %d out of range\n", idx); error = ERANGE; @@ -1224,7 +1226,8 @@ ena_sysctl_rss_indir_table(SYSCTL_HANDLER_ARGS) ena_rss_indir_get(adapter, indir->table); /* At this point msg has been clobbered by sysctl_handle_string. */ - ena_rss_copy_indir_buf(msg, indir->table); + ena_rss_copy_indir_buf(msg, indir->table, indir->table_size, + indir->record_size); if (error == 0) error = ena_rss_indir_set(adapter, indir->table); @@ -1234,4 +1237,17 @@ unlock: return (error); } + +static int +ena_sysctl_rss_indir_table_size(SYSCTL_HANDLER_ARGS) +{ + struct ena_adapter *adapter = arg1; + int actual_size; + + actual_size = ena_get_rss_indirection_table_size(adapter); + if (actual_size == 0) + return (EOPNOTSUPP); + + return sysctl_handle_int(oidp, &actual_size, 0, req); +} #endif /* RSS */