From nobody Sun Aug 30 00:26:51 2026 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hXXxR4Ny8z6pjhP for ; Sun, 30 Aug 2026 00:26:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hXXxR2sLVz3XT3 for ; Sun, 30 Aug 2026 00:26:51 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1788049611; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=sYLqufGBpGAsxvYAv1vOVg7G4D4oYPZlssCRIhrc/kQ=; b=RtwhlbnZ9J2qLGhcza4CZmtIBu55BGSKNSooT7Pyy9gS0mtWDGg0uz4NBlJ7n+uZ2M7OKm 74DRNhlec4Qk6D6VGu7+Wpy4VMG8Nse/FhvuNIZQy1yi43Ja5ITg8eckDJ5vTSJmx44kIE 9aboEkJB6sVwXBwVu1ctPs1cicAcYOvdYTlGjptIQeHLJG6tvw8mlzlGktPGTYs9rIANY1 YSHoN/iVtYIiNE85fgQh1ieikgchI5Xq4ESFJOoJW4fhsw5gga6ylaRoLN7BdzuToUrxfA 3+gm2/q0btcwoGRM1Dt+DWwTgBlvO8MhVUAtkVlhAwGSGfUpSxbxnAtwTP8ZWw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1788049611; a=rsa-sha256; cv=none; b=nOP42qs8pFBIugNqGKO4rsSASFmhZjxM35hcf1Mv61+xMm/4/fH2re8ngRbcqIsDjh0aSJ okhS83v165ioYTq7+Mt9vNp0MczQGLuJ1Ky5pBgvL8HyPUCSpopwQBmmET/hUfuEycsMVV wwxTM/KTOXab9BNb6CnjDzXoYVeFOp6xTu4TzIdjv0VbNU+8J5B42ChFbIKIOgKC2D9dP0 DN7aFXxTsWOr1EUA5B3PNIl4Aboxvy+iuH+qTYQrPOspCpLOGIH/CQUVVobypqOlG2sUTI Yx/KS9Og8IRMWxvwCHsKNkFI7mz3rWlyp14lU4jYzhU/owsD4cn1HQhv5K1v4g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1788049611; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=sYLqufGBpGAsxvYAv1vOVg7G4D4oYPZlssCRIhrc/kQ=; b=FU6YPgHAwLsnuXmgJFkC3+faFwi8FVHt2bD5vHD/TH8rNZ0+Kbg/z1X4Fr6c++pTjM0ajB EgfazXW2eiCKeLHR+pfPZ7V+MsnS0wKApUU9Mvc7Pw0t7ZRE2UO8zOAXcR3r7pOGUlrnkV 3Jx8fbX7Q4Icg2AukH/GVNde9nFPFZOknrE172LT5Sf4C6pb8p8Sr33D9g0BnL9xSlMdJJ 0fX/RskmhFYHVlOWEGG/wivpCoQ+D4nDNibqkQckI/74XB+v+ohNhMEKKAsOPDT91PNwAl zpZUcAOtFBhSxitFDoX1+SmYRXf4wK8v9OTjY5tmtGzGhjRePt79PB7B2YjZXQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hXXxR0jxLz12RQ for ; Sun, 30 Aug 2026 00:26:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 38220 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Sun, 30 Aug 2026 00:26:51 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kevin Bowling Subject: git: 4f44e80e7025 - stable/15 - ixgbe: Enable per-pool RSS on X550 family devices List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-branches@freebsd.org Sender: owner-dev-commits-src-branches@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kbowling X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: 4f44e80e70254a7f8fca75b5d739b06fe0b7545d Auto-Submitted: auto-generated Date: Sun, 30 Aug 2026 00:26:51 +0000 Message-Id: <6a9378cb.38220.dbd9ce1@gitrepo.freebsd.org> The branch stable/15 has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=4f44e80e70254a7f8fca75b5d739b06fe0b7545d commit 4f44e80e70254a7f8fca75b5d739b06fe0b7545d Author: Kevin Bowling AuthorDate: 2026-08-10 15:55:41 +0000 Commit: Kevin Bowling CommitDate: 2026-08-30 00:26:39 +0000 ixgbe: Enable per-pool RSS on X550 family devices X550 family devices provide a separate RSS key, redirection table, and MRQC register for every VMDq pool. With SR-IOV enabled, the driver continued programming only the global RSS state and never selected MRQC.MULTIPLE_RSS. VF-local RSS programming was therefore ineffective. Enable multiple-RSS mode for X550, X552, X553, and E610. Initialize the PF pool's 64-entry key, redirection table, and RSS hash controls. Leave each VF pool untouched so its driver retains ownership of its RSS key and mapping. E610 folds IPv6 extension-header traffic into its base RSS selectors and reserves the legacy EX selector bits. Translate those requested hash types rather than programming reserved bits. With two E610 VFs active and four PF queue sets, eight fixed TCP flows distributed across all four PF receive queues. E610 uses the same per-pool mode according to the E610 Datasheet, sections 7.1.3.6.2 and 8.2.2.8.20-21. Sponsored by: Dirk-Willem van Gulik from Web Weaving (E610 hardware) Sponsored by: BBOX.io (cherry picked from commit 87406f8351664aa444851499cda83b00fdc07662) --- sys/dev/ixgbe/if_ix.c | 89 +++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 69 insertions(+), 20 deletions(-) diff --git a/sys/dev/ixgbe/if_ix.c b/sys/dev/ixgbe/if_ix.c index dfca3e59976e..f08405059677 100644 --- a/sys/dev/ixgbe/if_ix.c +++ b/sys/dev/ixgbe/if_ix.c @@ -652,10 +652,13 @@ static void ixgbe_initialize_rss_mapping(struct ixgbe_softc *sc) { struct ixgbe_hw *hw = &sc->hw; - u32 reta = 0, mrqc, rss_key[10]; - int queue_id, reta_queues, table_size, index_mult; - int i, j; + bool per_pool_rss; + u32 reta = 0, mrqc, rss_fields, rss_key[10]; + int i, index_mult, j, queue_id, reta_queues, table_size; u32 rss_hash_config; +#ifdef PCI_IOV + u32 pfmrqc; +#endif if (sc->feat_en & IXGBE_FEATURE_RSS) { /* Fetch the configured RSS key */ @@ -667,6 +670,7 @@ ixgbe_initialize_rss_mapping(struct ixgbe_softc *sc) /* Set multiplier for RETA setup and table size based on MAC */ index_mult = 0x1; + per_pool_rss = false; table_size = 128; switch (sc->hw.mac.type) { case ixgbe_mac_82598EB: @@ -677,6 +681,12 @@ ixgbe_initialize_rss_mapping(struct ixgbe_softc *sc) case ixgbe_mac_X550EM_a: case ixgbe_mac_E610: table_size = 512; +#ifdef PCI_IOV + if (sc->iov_mode != IXGBE_NO_VM) { + per_pool_rss = true; + table_size = 64; + } +#endif break; default: break; @@ -684,13 +694,16 @@ ixgbe_initialize_rss_mapping(struct ixgbe_softc *sc) /* * The global RETA is shared by the PF and VFs on 82599 and X540. - * Program all four queue indices while SR-IOV is active so a VF can - * use its full queue grant even when the PF uses fewer queues. - * PSRTYPE.RQPL limits the subset selected within each pool. + * X550-family devices instead use per-pool tables in multiple-RSS + * mode; initialize only the PF pool here because each VF owns and + * programs its own key and redirection table. + * On the shared tables, program all four queue indices while SR-IOV + * is active so a VF can use its full queue grant even when the PF uses + * fewer queues. PSRTYPE.RQPL limits the subset selected in each pool. */ reta_queues = sc->num_rx_queues; #ifdef PCI_IOV - if (sc->iov_mode != IXGBE_NO_VM) + if (sc->iov_mode != IXGBE_NO_VM && !per_pool_rss) reta_queues = MAX(reta_queues, 4); #endif @@ -717,7 +730,12 @@ ixgbe_initialize_rss_mapping(struct ixgbe_softc *sc) reta = reta >> 8; reta = reta | (((uint32_t)queue_id) << 24); if ((i & 3) == 3) { - if (i < 128) + if (per_pool_rss) { +#ifdef PCI_IOV + IXGBE_WRITE_REG(hw, + IXGBE_PFVFRETA(i >> 2, sc->pool), reta); +#endif + } else if (i < 128) IXGBE_WRITE_REG(hw, IXGBE_RETA(i >> 2), reta); else IXGBE_WRITE_REG(hw, @@ -727,8 +745,15 @@ ixgbe_initialize_rss_mapping(struct ixgbe_softc *sc) } /* Now fill our hash function seeds */ - for (i = 0; i < 10; i++) - IXGBE_WRITE_REG(hw, IXGBE_RSSRK(i), rss_key[i]); + for (i = 0; i < 10; i++) { + if (per_pool_rss) { +#ifdef PCI_IOV + IXGBE_WRITE_REG(hw, IXGBE_PFVFRSSRK(i, sc->pool), + rss_key[i]); +#endif + } else + IXGBE_WRITE_REG(hw, IXGBE_RSSRK(i), rss_key[i]); + } /* Perform hash on these packet types */ if (sc->feat_en & IXGBE_FEATURE_RSS) @@ -748,26 +773,50 @@ ixgbe_initialize_rss_mapping(struct ixgbe_softc *sc) } mrqc = ixgbe_get_mrqc(sc->iov_mode); + rss_fields = 0; if (rss_hash_config & RSS_HASHTYPE_RSS_IPV4) - mrqc |= IXGBE_MRQC_RSS_FIELD_IPV4; + rss_fields |= IXGBE_MRQC_RSS_FIELD_IPV4; if (rss_hash_config & RSS_HASHTYPE_RSS_TCP_IPV4) - mrqc |= IXGBE_MRQC_RSS_FIELD_IPV4_TCP; + rss_fields |= IXGBE_MRQC_RSS_FIELD_IPV4_TCP; if (rss_hash_config & RSS_HASHTYPE_RSS_IPV6) - mrqc |= IXGBE_MRQC_RSS_FIELD_IPV6; + rss_fields |= IXGBE_MRQC_RSS_FIELD_IPV6; if (rss_hash_config & RSS_HASHTYPE_RSS_TCP_IPV6) - mrqc |= IXGBE_MRQC_RSS_FIELD_IPV6_TCP; + rss_fields |= IXGBE_MRQC_RSS_FIELD_IPV6_TCP; if (rss_hash_config & RSS_HASHTYPE_RSS_IPV6_EX) - mrqc |= IXGBE_MRQC_RSS_FIELD_IPV6_EX; + rss_fields |= IXGBE_MRQC_RSS_FIELD_IPV6_EX; if (rss_hash_config & RSS_HASHTYPE_RSS_TCP_IPV6_EX) - mrqc |= IXGBE_MRQC_RSS_FIELD_IPV6_EX_TCP; + rss_fields |= IXGBE_MRQC_RSS_FIELD_IPV6_EX_TCP; if (rss_hash_config & RSS_HASHTYPE_RSS_UDP_IPV4) - mrqc |= IXGBE_MRQC_RSS_FIELD_IPV4_UDP; + rss_fields |= IXGBE_MRQC_RSS_FIELD_IPV4_UDP; if (rss_hash_config & RSS_HASHTYPE_RSS_UDP_IPV6) - mrqc |= IXGBE_MRQC_RSS_FIELD_IPV6_UDP; + rss_fields |= IXGBE_MRQC_RSS_FIELD_IPV6_UDP; if (rss_hash_config & RSS_HASHTYPE_RSS_UDP_IPV6_EX) - mrqc |= IXGBE_MRQC_RSS_FIELD_IPV6_EX_UDP; + rss_fields |= IXGBE_MRQC_RSS_FIELD_IPV6_EX_UDP; + if (hw->mac.type == ixgbe_mac_E610) { + /* E610 folds IPv6 extension headers into the base selectors. */ + if (rss_fields & IXGBE_MRQC_RSS_FIELD_IPV6_EX) + rss_fields |= IXGBE_MRQC_RSS_FIELD_IPV6; + if (rss_fields & IXGBE_MRQC_RSS_FIELD_IPV6_EX_TCP) + rss_fields |= IXGBE_MRQC_RSS_FIELD_IPV6_TCP; + if (rss_fields & IXGBE_MRQC_RSS_FIELD_IPV6_EX_UDP) + rss_fields |= IXGBE_MRQC_RSS_FIELD_IPV6_UDP; + rss_fields &= ~(IXGBE_MRQC_RSS_FIELD_IPV6_EX | + IXGBE_MRQC_RSS_FIELD_IPV6_EX_TCP | + IXGBE_MRQC_RSS_FIELD_IPV6_EX_UDP); + } + + if (per_pool_rss) { +#ifdef PCI_IOV + mrqc |= IXGBE_MRQC_MULTIPLE_RSS; + IXGBE_WRITE_REG(hw, IXGBE_MRQC, mrqc); - IXGBE_WRITE_REG(hw, IXGBE_MRQC, mrqc); + pfmrqc = IXGBE_MRQC_RSSEN | rss_fields; + IXGBE_WRITE_REG(hw, IXGBE_PFVFMRQC(sc->pool), pfmrqc); +#endif + } else { + mrqc |= rss_fields; + IXGBE_WRITE_REG(hw, IXGBE_MRQC, mrqc); + } } /* ixgbe_initialize_rss_mapping */ /************************************************************************