From nobody Thu Oct 29 06:03:20 2026 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jFYZ05h6gz6xrd2 for ; Thu, 29 Oct 2026 06:03:20 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jFYZ02HLnz4sWn for ; Thu, 29 Oct 2026 06:03:20 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793253800; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=SW42fgJOw7692V2UszOO1rNNeHp1RsWAJpGg0d3tMGA=; b=vabyGCaxvI+aQE7qk0cOfriYnrs6Gk8iu6gAzbHiH/s8UOGddM8gOUkpz7j+LFBgupX+e2 iARp2dj3SWWlqgcEEXa++MG0ZDO/3aL854SB8oNwVdKggiD6A6K7JVubeJVdwFas4M6Gdo JRC0z+D0TTgj8VNXI08GC8EqUNbBIY4I6XfUFGgmtjtKTF4ZWRYEI6izECYSCdeJORp+I1 FHLwwQWQdAEcfEUqwzNMdeJq37wPybvBfpKKQGboEoPDwC4XfMFrBM+YceJBIlKKIH9GVt 8LMhVUQEkNA1VXMY38CtgB+IIhyOx5IaMg71uvCIbE8X1HItdy+sn5szB3th7Q== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1793253800; b=XYWsAKwXCepiu5SYNkaAiIFf6tA2pyWZPzHPb4N3JMgfKQGgo5U1SKlwHHnvG72fVVpI+C dwzAdGRDL6zgmqARUpzCDyfu5mLOgDHX2KxPoy0K6Q90yVDgtg0p42wXl0ODKlISI2L2wN BMwLHdq6czDzdQXPqcImKXqrZB63uzmRuu5jJ1Z/pEcKtjCk27i6ISBGBzgCIV6WRqSq98 eaprM2rsY2QO+ZZvoK9cLdoXZ51//0k+MU98QZtGY49tWC2dbXL+Lb1+jQI+ql7V2rx8rf RRJHC/v+zvvEWwBD0VNq5c9rVUoC/NXLYYe9Af6nphqBLTzx78aVJQp26MMo0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793253800; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=SW42fgJOw7692V2UszOO1rNNeHp1RsWAJpGg0d3tMGA=; b=xCUqdJwkNtFr+uQrhAgmPzW8Rzl02Wx7Rh3K+8p+ca+sXTHRl/ns+U4UXtXsZypgvELOnL DfF9sK5+V7OVOvVLLmyuWIkZbrDGlISCh3+XeYbQWhswpXM46pb7Q5a0B+3TOaElFmxz0F GZejmEliBlXwZB8woTG2/V+fTVXa1TQz45piLw6NX3F/qDyMTMtXX30lCEzBgg28S6Jkyt TR/GBfvwSkBDNeBTvBUh2nLsDksTqvsfJLCgvxswg5OoZq7Vy7CMofBmDWyoVUGgvcOm4o +Gx98PIj/A1utMdYB9aHIIyhgAMYdPDFhtbMMv9yUN2B3n3Scm3xUqr2iKPjaw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jFYZ01ygXzR83 for ; Thu, 29 Oct 2026 06:03:20 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 3a39d by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Thu, 29 Oct 2026 06:03:20 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org Cc: Andrew Gallatin From: Kevin Bowling Subject: git: e58dcd40db3a - stable/15 - iflib: implement driver-provided queue selection for simple tx List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches X-BeenThere: dev-commits-src-branches@freebsd.org Sender: owner-dev-commits-src-branches@FreeBSD.org List-Id: Commits to the stable branches of the FreeBSD src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kbowling X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: e58dcd40db3ab93d6cba9b99dbd2556643340449 Auto-Submitted: auto-generated Date: Thu, 29 Oct 2026 06:03:20 +0000 Message-Id: <6ae2e1a8.3a39d.57cb65c2@gitrepo.freebsd.org> The branch stable/15 has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=e58dcd40db3ab93d6cba9b99dbd2556643340449 commit e58dcd40db3ab93d6cba9b99dbd2556643340449 Author: Andrew Gallatin AuthorDate: 2026-09-16 15:41:24 +0000 Commit: Kevin Bowling CommitDate: 2026-10-29 05:38:10 +0000 iflib: implement driver-provided queue selection for simple tx Until now, simple tx has used its own queue selector, and has ignored isc_txq_select and isc_txq_select_v2 (not this only seems to matter for ice(4) with dcb enabled). This change makes simple-tx use isc_txq_select* when present. The implementation is defined to be efficient, with a transmit routine chosen up-front that hard-codes the queue selection and calls an always-inlined body. This avoids a useless test per packet in the hotpath, and also may avoid speculation into header parsing. Note that this was designed for readability and efficiency in the common case (interface up, not ALTQ). That's why we do queue selection without duplicating nic-running and altq checks, leaving them to the common implmentation. Sponsored by: Netflix Reviewed by: kbowling Differential Revision: https://reviews.freebsd.org/D59712 (cherry picked from commit c6728cf5850da500af177b2bb326dc29a32fc450) --- sys/net/iflib.c | 118 +++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 87 insertions(+), 31 deletions(-) diff --git a/sys/net/iflib.c b/sys/net/iflib.c index 37c7870fb45d..e639343b93f5 100644 --- a/sys/net/iflib.c +++ b/sys/net/iflib.c @@ -192,6 +192,8 @@ static void iru_init(if_rxd_update_t iru, iflib_rxq_t rxq, uint8_t flid); static void iflib_timer(void *arg); static void iflib_tqg_detach(if_ctx_t ctx); static int iflib_simple_transmit(if_t ifp, struct mbuf *m); +static int iflib_simple_transmit_txq_select(if_t ifp, struct mbuf *m); +static int iflib_simple_transmit_txq_select_v2(if_t ifp, struct mbuf *m); static void iflib_simple_if_start(if_t ifp); static void iflib_simple_txq_drain(iflib_txq_t txq); @@ -4759,7 +4761,6 @@ iflib_altq_if_start(if_t ifp) static int iflib_altq_if_transmit(if_t ifp, struct mbuf *m) { - if_ctx_t ctx = if_getsoftc(ifp); int err; if (if_altq_is_enabled(ifp)) { @@ -4768,12 +4769,7 @@ iflib_altq_if_transmit(if_t ifp, struct mbuf *m) if_start(ifp); return (err); } - if (ctx->ifc_sysctl_simple_tx) - err = iflib_simple_transmit(ifp, m); - else - err = iflib_if_transmit(ifp, m); - - return (err); + return (iflib_if_transmit(ifp, m)); } #endif /* ALTQ */ @@ -5569,9 +5565,7 @@ iflib_device_register(device_t dev, void *sc, if_shared_ctx_t sctx, if_ctx_t *ct ifp = ctx->ifc_ifp; if (ctx->ifc_sysctl_simple_tx) { /* if_start drives the same drbr drain when ALTQ is active. */ -#ifndef ALTQ if_settransmitfn(ifp, iflib_simple_transmit); -#endif if_setstartfn(ifp, iflib_simple_if_start); device_printf(dev, "using simple transmit\n"); } @@ -5588,6 +5582,19 @@ iflib_device_register(device_t dev, void *sc, if_shared_ctx_t sctx, if_ctx_t *ct fail_cleanup); _iflib_pre_assert(scctx); ctx->ifc_txrx = *scctx->isc_txrx; + /* + * Bind optional queue selection to a transmit entry point once, rather + * than testing for optional methods for every packet. Prefer v2 when + * the driver provides both methods. + */ + if (ctx->ifc_sysctl_simple_tx) { + if (ctx->isc_txq_select_v2 != NULL) + if_settransmitfn(ifp, + iflib_simple_transmit_txq_select_v2); + else if (ctx->isc_txq_select != NULL) + if_settransmitfn(ifp, + iflib_simple_transmit_txq_select); + } MPASS(scctx->isc_dma_width <= flsll(BUS_SPACE_MAXADDR)); @@ -7993,23 +8000,6 @@ iflib_debugnet_poll(if_t ifp, int count) } #endif /* DEBUGNET */ -static inline iflib_txq_t -iflib_simple_select_queue(if_ctx_t ctx, struct mbuf *m) -{ - int qidx; - -#ifdef ALTQ - /* ALTQ-enabled interfaces always use queue 0. */ - if (if_altq_is_enabled(ctx->ifc_ifp)) - return (&ctx->ifc_txqs[0]); -#endif - if ((NTXQSETS(ctx) > 1) && M_HASHTYPE_GET(m)) - qidx = QIDX(ctx, m); - else - qidx = NTXQSETS(ctx) + FIRST_QSET(ctx) - 1; - return (&ctx->ifc_txqs[qidx]); -} - enum iflib_txq_producer_status { IFLIB_TXQ_PRODUCER_ENTERED, IFLIB_TXQ_PRODUCER_QUIESCING, @@ -8270,22 +8260,34 @@ iflib_simple_transmit_locked(iflib_txq_t txq, struct mbuf *m, int *bytes, return (error); } -static int -iflib_simple_transmit(if_t ifp, struct mbuf *m) +/* + * Always inline the common transmit path so queue selection does not add a + * function call to the default simple transmit path. + */ +static __always_inline int +iflib_simple_transmit_impl(if_ctx_t ctx, if_t ifp, struct mbuf *m, + iflib_txq_t txq) { - if_ctx_t ctx; - iflib_txq_t txq; struct mbuf **m_defer; enum iflib_txq_producer_status producer_status; bool pinned; int error, i, reclaimable; int bytes_sent = 0, pkt_sent = 0, mcast_sent = 0; + +#ifdef ALTQ + if (if_altq_is_enabled(ifp)) { + IFQ_ENQUEUE(&ifp->if_snd, m, error); /* XXX - DRVAPI */ + if (error == 0) + if_start(ifp); + return (error); + } +#endif + ctx = if_getsoftc(ifp); if (__predict_false(!iflib_is_running(ctx) || !LINK_ACTIVE(ctx))) goto net_down; - txq = iflib_simple_select_queue(ctx, m); /* * Avoid blocking behind another transmitter; the ring is drained by * whoever holds ift_mtx, by tx completion, or by the watchdog timer. @@ -8360,6 +8362,60 @@ net_down: return (ENETDOWN); } +static int +iflib_simple_transmit(if_t ifp, struct mbuf *m) +{ + if_ctx_t ctx; + iflib_txq_t txq; + int qidx; + + ctx = if_getsoftc(ifp); + if ((NTXQSETS(ctx) > 1) && M_HASHTYPE_GET(m)) + qidx = QIDX(ctx, m); + else + qidx = NTXQSETS(ctx) + FIRST_QSET(ctx) - 1; + MPASS(qidx < NTXQSETS(ctx)); + txq = &ctx->ifc_txqs[qidx]; + return (iflib_simple_transmit_impl(ctx, ifp, m, txq)); +} + +static int +iflib_simple_transmit_txq_select(if_t ifp, struct mbuf *m) +{ + if_ctx_t ctx; + int qidx; + + ctx = if_getsoftc(ifp); + qidx = ctx->isc_txq_select(ctx->ifc_softc, m); + MPASS(qidx < NTXQSETS(ctx)); + return (iflib_simple_transmit_impl(ctx, ifp, m, + &ctx->ifc_txqs[qidx])); +} + +static int +iflib_simple_transmit_txq_select_v2(if_t ifp, struct mbuf *m) +{ + struct if_pkt_info pi; + if_ctx_t ctx; + uint64_t early_pullups = 0; + int error, qidx; + + ctx = if_getsoftc(ifp); + memset(&pi, 0, sizeof(pi)); + error = iflib_parse_header_partial(&pi, &m, &early_pullups); + if (error != 0) { + /* Assign pullups for bad packets to the default queue. */ + ctx->ifc_txqs[0].ift_pullups += early_pullups; + DBG_COUNTER_INC(encap_txd_encap_fail); + return (error); + } + qidx = ctx->isc_txq_select_v2(ctx->ifc_softc, m, &pi); + MPASS(qidx < NTXQSETS(ctx)); + ctx->ifc_txqs[qidx].ift_pullups += early_pullups; + return (iflib_simple_transmit_impl(ctx, ifp, m, + &ctx->ifc_txqs[qidx])); +} + /* * ALTQ entry point. drbr_dequeue() pulls from ifp->if_snd when a discipline * is attached, so the drain loop is shared with the if_transmit path. Only