From nobody Wed Sep 16 15:58:40 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hlNpm5K7hz6rhD3 for ; Wed, 16 Sep 2026 15:58:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hlNpm3Jb9z4pvV for ; Wed, 16 Sep 2026 15:58:40 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1789574320; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=X0uBbvGOVqk/i8+pybS5KT9v9gIidzG/9FoHw30XGyI=; b=gCdibYpbVq3g874G+bV+YDXx802UaD1znWG1j5wbtkVB/NrCplrUQBhPhnIKg/so4dR9CO /6/Gkxafg0W8xcM+36Q6KAMl40wAiM+bNOOtGt51pvNbYzsHl08gZtURr/Y8lTjcOw+259 7tzeyT9fRiqVa8F2mbW0lNTEcWRPIy2T+thqZUC3ol2UXr1YaYP8kFSLliV7/Lk6rQWlJz MOjkGNCMZxwhEBgp+SoI4g0Nbk7gsriynIPNc01FncXaEvgx9mBWNHr01tQfxljS6+AA1A VIE06rMTE2eA5qgCHnY3o4pbjQZ5b1C/YAZlJ/F2qdwUGccAmdS4+v5BPO9t9w== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1789574320; b=C5vvTCe50x2D6WVBUC6hi9wvzdrxgFnN97qOXeEs22XDRmqrbygbwBkOIFnkPYBra0TsH7 +nstgknJOBBVqJSyFbZJW1UjAbITdaJJer6JfhELLIl2ZXoKOnD6CNUlO7jbu3K6NOXLqx RA1UG0lbIBVKLgl+mShsxC3SJZdgHBHAJA0WkPET6pmU0bJqkG1o8z1C0SOoeKS0PH7/1V wOKED3Oy27/M3W106m4PkPMiqyZ6j6xxtkTCXeyPSesbNFuZ8njaUrqkhnQ05XVvHz2Ici Zr33Q7baRkZ0zR4HqXTveRBdL1bDt8YDMLLyEHp9FRJRCC9jODj+cIiZa/aYPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1789574320; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=X0uBbvGOVqk/i8+pybS5KT9v9gIidzG/9FoHw30XGyI=; b=Osb1prvQVohmnPv7NXXHVjmfFh5O5TRrcjdefD1KdchpZK5Z4/FaX3CFc95TZjtDUYFNz+ RF86OvQM3Uv4uqDmqXRU6fU8xQZJylRYyIpGS6YeKSWqdcFYdgrR00Na08D0vACUVYJK2m 2mgfcPQdvZvViy02y84j3apMFjzlGhN9F9ci13+I8wOPHPWz2xhia8KVnnEKbW5z3yyOpB qPp+3//QQ6WdKixyjp80+cSq+oNSKmol7dOQnboM7HOSZU/QlCzIU1YdulKZ7wd7xSsOGX ukrVbUOy+fV1a245RLP7KMGZLSf8FvhvkxfOmBc9uGlauVVtTYe6fs2gY+nHYw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hlNpm2LhXzskk for ; Wed, 16 Sep 2026 15:58:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 1978e by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Wed, 16 Sep 2026 15:58:40 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Andrew Gallatin Subject: git: c6728cf5850d - main - iflib: implement driver-provided queue selection for simple tx List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: gallatin X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: c6728cf5850da500af177b2bb326dc29a32fc450 Auto-Submitted: auto-generated Date: Wed, 16 Sep 2026 15:58:40 +0000 Message-Id: <6aaabcb0.1978e.414711b5@gitrepo.freebsd.org> The branch main has been updated by gallatin: URL: https://cgit.FreeBSD.org/src/commit/?id=c6728cf5850da500af177b2bb326dc29a32fc450 commit c6728cf5850da500af177b2bb326dc29a32fc450 Author: Andrew Gallatin AuthorDate: 2026-09-16 15:41:24 +0000 Commit: Andrew Gallatin CommitDate: 2026-09-16 15:54:29 +0000 iflib: implement driver-provided queue selection for simple tx Until now, simple tx has used its own queue selector, and has ignored isc_txq_select and isc_txq_select_v2 (not this only seems to matter for ice(4) with dcb enabled). This change makes simple-tx use isc_txq_select* when present. The implementation is defined to be efficient, with a transmit routine chosen up-front that hard-codes the queue selection and calls an always-inlined body. This avoids a useless test per packet in the hotpath, and also may avoid speculation into header parsing. Note that this was designed for readability and efficiency in the common case (interface up, not ALTQ). That's why we do queue selection without duplicating nic-running and altq checks, leaving them to the common implmentation. Sponsored by: Netflix Reviewed by: kbowling Differential Revision: https://reviews.freebsd.org/D59712 --- sys/net/iflib.c | 118 +++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 87 insertions(+), 31 deletions(-) diff --git a/sys/net/iflib.c b/sys/net/iflib.c index 334160dbc850..2257e64a5a8e 100644 --- a/sys/net/iflib.c +++ b/sys/net/iflib.c @@ -193,6 +193,8 @@ static void iru_init(if_rxd_update_t iru, iflib_rxq_t rxq, uint8_t flid); static void iflib_timer(void *arg); static void iflib_tqg_detach(if_ctx_t ctx); static int iflib_simple_transmit(if_t ifp, struct mbuf *m); +static int iflib_simple_transmit_txq_select(if_t ifp, struct mbuf *m); +static int iflib_simple_transmit_txq_select_v2(if_t ifp, struct mbuf *m); static void iflib_simple_if_start(if_t ifp); static void iflib_simple_txq_drain(iflib_txq_t txq); @@ -4749,7 +4751,6 @@ iflib_altq_if_start(if_t ifp) static int iflib_altq_if_transmit(if_t ifp, struct mbuf *m) { - if_ctx_t ctx = if_getsoftc(ifp); int err; if (if_altq_is_enabled(ifp)) { @@ -4758,12 +4759,7 @@ iflib_altq_if_transmit(if_t ifp, struct mbuf *m) if_start(ifp); return (err); } - if (ctx->ifc_sysctl_simple_tx) - err = iflib_simple_transmit(ifp, m); - else - err = iflib_if_transmit(ifp, m); - - return (err); + return (iflib_if_transmit(ifp, m)); } #endif /* ALTQ */ @@ -5586,9 +5582,7 @@ iflib_device_register(device_t dev, void *sc, if_shared_ctx_t sctx, if_ctx_t *ct ifp = ctx->ifc_ifp; if (ctx->ifc_sysctl_simple_tx) { /* if_start drives the same drbr drain when ALTQ is active. */ -#ifndef ALTQ if_settransmitfn(ifp, iflib_simple_transmit); -#endif if_setstartfn(ifp, iflib_simple_if_start); device_printf(dev, "using simple transmit\n"); } @@ -5605,6 +5599,19 @@ iflib_device_register(device_t dev, void *sc, if_shared_ctx_t sctx, if_ctx_t *ct fail_cleanup); _iflib_pre_assert(scctx); ctx->ifc_txrx = *scctx->isc_txrx; + /* + * Bind optional queue selection to a transmit entry point once, rather + * than testing for optional methods for every packet. Prefer v2 when + * the driver provides both methods. + */ + if (ctx->ifc_sysctl_simple_tx) { + if (ctx->isc_txq_select_v2 != NULL) + if_settransmitfn(ifp, + iflib_simple_transmit_txq_select_v2); + else if (ctx->isc_txq_select != NULL) + if_settransmitfn(ifp, + iflib_simple_transmit_txq_select); + } MPASS(scctx->isc_dma_width <= flsll(BUS_SPACE_MAXADDR)); @@ -8016,23 +8023,6 @@ iflib_debugnet_poll(if_t ifp, int count) } #endif /* DEBUGNET */ -static inline iflib_txq_t -iflib_simple_select_queue(if_ctx_t ctx, struct mbuf *m) -{ - int qidx; - -#ifdef ALTQ - /* ALTQ-enabled interfaces always use queue 0. */ - if (if_altq_is_enabled(ctx->ifc_ifp)) - return (&ctx->ifc_txqs[0]); -#endif - if ((NTXQSETS(ctx) > 1) && M_HASHTYPE_GET(m)) - qidx = QIDX(ctx, m); - else - qidx = NTXQSETS(ctx) + FIRST_QSET(ctx) - 1; - return (&ctx->ifc_txqs[qidx]); -} - enum iflib_txq_producer_status { IFLIB_TXQ_PRODUCER_ENTERED, IFLIB_TXQ_PRODUCER_QUIESCING, @@ -8293,22 +8283,34 @@ iflib_simple_transmit_locked(iflib_txq_t txq, struct mbuf *m, int *bytes, return (error); } -static int -iflib_simple_transmit(if_t ifp, struct mbuf *m) +/* + * Always inline the common transmit path so queue selection does not add a + * function call to the default simple transmit path. + */ +static __always_inline int +iflib_simple_transmit_impl(if_ctx_t ctx, if_t ifp, struct mbuf *m, + iflib_txq_t txq) { - if_ctx_t ctx; - iflib_txq_t txq; struct mbuf **m_defer; enum iflib_txq_producer_status producer_status; bool pinned; int error, i, reclaimable; int bytes_sent = 0, pkt_sent = 0, mcast_sent = 0; + +#ifdef ALTQ + if (if_altq_is_enabled(ifp)) { + IFQ_ENQUEUE(&ifp->if_snd, m, error); /* XXX - DRVAPI */ + if (error == 0) + if_start(ifp); + return (error); + } +#endif + ctx = if_getsoftc(ifp); if (__predict_false(!iflib_is_running(ctx) || !LINK_ACTIVE(ctx))) goto net_down; - txq = iflib_simple_select_queue(ctx, m); /* * Avoid blocking behind another transmitter; the ring is drained by * whoever holds ift_mtx, by tx completion, or by the watchdog timer. @@ -8383,6 +8385,60 @@ net_down: return (ENETDOWN); } +static int +iflib_simple_transmit(if_t ifp, struct mbuf *m) +{ + if_ctx_t ctx; + iflib_txq_t txq; + int qidx; + + ctx = if_getsoftc(ifp); + if ((NTXQSETS(ctx) > 1) && M_HASHTYPE_GET(m)) + qidx = QIDX(ctx, m); + else + qidx = NTXQSETS(ctx) + FIRST_QSET(ctx) - 1; + MPASS(qidx < NTXQSETS(ctx)); + txq = &ctx->ifc_txqs[qidx]; + return (iflib_simple_transmit_impl(ctx, ifp, m, txq)); +} + +static int +iflib_simple_transmit_txq_select(if_t ifp, struct mbuf *m) +{ + if_ctx_t ctx; + int qidx; + + ctx = if_getsoftc(ifp); + qidx = ctx->isc_txq_select(ctx->ifc_softc, m); + MPASS(qidx < NTXQSETS(ctx)); + return (iflib_simple_transmit_impl(ctx, ifp, m, + &ctx->ifc_txqs[qidx])); +} + +static int +iflib_simple_transmit_txq_select_v2(if_t ifp, struct mbuf *m) +{ + struct if_pkt_info pi; + if_ctx_t ctx; + uint64_t early_pullups = 0; + int error, qidx; + + ctx = if_getsoftc(ifp); + memset(&pi, 0, sizeof(pi)); + error = iflib_parse_header_partial(&pi, &m, &early_pullups); + if (error != 0) { + /* Assign pullups for bad packets to the default queue. */ + ctx->ifc_txqs[0].ift_pullups += early_pullups; + DBG_COUNTER_INC(encap_txd_encap_fail); + return (error); + } + qidx = ctx->isc_txq_select_v2(ctx->ifc_softc, m, &pi); + MPASS(qidx < NTXQSETS(ctx)); + ctx->ifc_txqs[qidx].ift_pullups += early_pullups; + return (iflib_simple_transmit_impl(ctx, ifp, m, + &ctx->ifc_txqs[qidx])); +} + /* * ALTQ entry point. drbr_dequeue() pulls from ifp->if_snd when a discipline * is attached, so the drain loop is shared with the if_transmit path. Only