From nobody Thu Oct 29 06:03:12 2026 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jFYYr6HY0z6xrFr for ; Thu, 29 Oct 2026 06:03:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jFYYr4Q9Yz4sKW for ; Thu, 29 Oct 2026 06:03:12 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793253792; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=9qlKNO4ebaRh5O7z4ghgzIK8SrF/cuOL9aWo3DEeO2g=; b=hYQHW7NdHf5gA3mzxkZw4TmYMZ9yUD8aDvGBFyAfy0dLFKMNUOqmt1gntFwCOMhT0T+RdW khbcGoROk0UYkmsm/Y8qxT4M8vF2b6nrKaacy4C2ULZ/9VzIr62bH5b3pTTH5mNO0QrNeG t15tcMz1AMpE17UAvCIxspT03oNbtxnvTtN1RzMYZgT9STL05J2UMwrWIEflZruTsC/Wf2 eFUeEjpdw9bybSaO1lt3aHQvpzJHEJkFNX4TGbFmhgzk2ew29PsXMVVf/09c3ole0IhnCx EdhAMTvCWJSRGIWYl5dBdytvmYwj/FKFfv7CJ2FClzyavjoHisKcebQl4Pme+Q== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1793253792; b=Cv0szRa5pUiMIgvqRAk7hjRVAbEOy0llsWVxABC2/JgBLkAAf00IL7CQAmEgH+FWxyhUZM Y90cvL915gMeTQoLoO0grhWo+/WOCUDofSxGhVNUb+Zb9R3Vp0qQlrMzRZFs0Ul5wybW55 78sppNZfjPxhVno19Iz/Bu2E/ts9roe2TJZtYXuECjeDoBHUZX4obyT5I0STTYIJQWgQAe EQqjnBC3NTYh9aJ8OsHcFd6aFdi3upPJDXac3W8fqp0ugyhU3FcBK1I+49ZAY0EwKpp7ol +TbnMufCYHhV6hpaUZGjCQ1c9iJEHWqoNvmzf7WHMzuO7LL4YsvHyf6OVm56Vw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793253792; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=9qlKNO4ebaRh5O7z4ghgzIK8SrF/cuOL9aWo3DEeO2g=; b=PoR3jF+2eLNnzYdnO5qR4LTQj0DF5gLDfQUHkJ6t9o5mBCc8b9m2pzcDmapIA3iHDcEGR4 OhdbdKXmil5lRt20bDn+B9ey8oJNh7wG82AWAze+g26Z6sVb5zV6m8fzpz1A/C3NbLE+Od jVWxMOdH7VvXZk2Dn/+UBgM5qqMe4R8qmCixxo/GoqsUU545uHtYesYIuYir4Nj/dpqRI8 7SdoKzT3qlZR9b1w0I1P1Ue64DED2BRbyhMCSUDsQcdFVB+wfGsSCX7sJJgF9pZt4MYyyU k443/FcuS6LHn1/mFR/8Va0lBr0RpjKNtzIvJ05fJTNPFZn+KQXwJ80ymzWLDA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jFYYr3wXWzR5r for ; Thu, 29 Oct 2026 06:03:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 3acbe by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Thu, 29 Oct 2026 06:03:12 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kevin Bowling Subject: git: 423c6190ac67 - stable/15 - iflib: Require sustained demand for TX watchdog List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches X-BeenThere: dev-commits-src-branches@freebsd.org Sender: owner-dev-commits-src-branches@FreeBSD.org List-Id: Commits to the stable branches of the FreeBSD src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kbowling X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: 423c6190ac673589978bb4ff33d80cdf56a14695 Auto-Submitted: auto-generated Date: Thu, 29 Oct 2026 06:03:12 +0000 Message-Id: <6ae2e1a0.3acbe.4979e9b6@gitrepo.freebsd.org> The branch stable/15 has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=423c6190ac673589978bb4ff33d80cdf56a14695 commit 423c6190ac673589978bb4ff33d80cdf56a14695 Author: Kevin Bowling AuthorDate: 2026-08-27 02:00:34 +0000 Commit: Kevin Bowling CommitDate: 2026-10-29 05:28:38 +0000 iflib: Require sustained demand for TX watchdog The restored watchdog arms when the outstanding descriptor count grows, but then continues counting based only on the queue remaining frozen. A single growth sample can therefore leave a quiet, nearly empty queue armed until the watchdog resets the interface. Lockless sampling of the queue counters can also manufacture the initial growth sample. This matches watchdog reports from I354 queues with 979 or 980 of 1022 usable descriptors still available. Neither queue was under transmit backpressure when the reset flapped its link. Keep the watchdog armed only while the outstanding count continues to grow, the software ring is stalled, or the hardware ring is at iflib's backpressure threshold. The last condition preserves hang detection with simple-TX, which bypasses the software ring. A busy hang still reaches the verdict while a frozen but quiet tail disarms. Retain the final driver completion peek so a missed completion interrupt schedules the queue task instead of resetting it. Validated on an 82580 with one and four queue sets in the default mp_ring and simple-TX modes. Sustained traffic and repeated burst/idle cycles produced no false resets. Sixteen-flow runs exercised all four queues in both modes. Clearing TCTL.EN under load in each configuration filled the rings; the reset counter advanced once per injection, reset restored TCTL and the link, and traffic recovered. Tested by: glebius Reviewed by: iflib (gallatin), manpages (ziaee) Fixes: 69c3e0de01c1 ("iflib: restore TX watchdog functionality") Sponsored by: BBOX.io (cherry picked from commit a550e8280b9881c8c207b842f51065f155b84951) --- share/man/man4/iflib.4 | 14 +++++++----- sys/net/iflib.c | 60 ++++++++++++++++++++++++++++---------------------- 2 files changed, 43 insertions(+), 31 deletions(-) diff --git a/share/man/man4/iflib.4 b/share/man/man4/iflib.4 index dba07412088d..08c7682f253d 100644 --- a/share/man/man4/iflib.4 +++ b/share/man/man4/iflib.4 @@ -1,4 +1,4 @@ -.Dd August 4, 2026 +.Dd August 26, 2026 .Dt IFLIB 4 .Os .Sh NAME @@ -131,10 +131,14 @@ Setting this to a non-zero value will disable the use of this feature. .It Va net.iflib.tx_watchdog_periods Number of consecutive .Va net.iflib.timer_default -intervals for which a transmit queue must stay frozen, -that is, hold descriptors the hardware has not reported as completed -while none are reclaimed, -before iflib asks the hardware whether it has completions pending. +intervals for which a transmit queue must remain frozen under demand before +.Nm +asks the hardware whether it has completions pending. +A queue is frozen when it holds descriptors the hardware has not reported as +completed while none are reclaimed. +Demand persists while the outstanding descriptor count grows, the software +queue is stalled, or the hardware descriptor ring is at its backpressure +threshold. If it reports none, the interface is reset. Setting this to zero disables the check. .El diff --git a/sys/net/iflib.c b/sys/net/iflib.c index 515bde6c7e26..2829e9992a03 100644 --- a/sys/net/iflib.c +++ b/sys/net/iflib.c @@ -515,6 +515,9 @@ get_inuse(int size, qidx_t cidx, qidx_t pidx, uint8_t gen) #define TXQ_AVAIL(txq) ((txq->ift_size - txq->ift_pad) -\ get_inuse(txq->ift_size, txq->ift_cidx, txq->ift_pidx, txq->ift_gen)) +#define MAX_TX_DESC(ctx) MAX((ctx)->ifc_softc_ctx.isc_tx_tso_segments_max, \ + (ctx)->ifc_softc_ctx.isc_tx_nsegments) + #define IDXDIFF(head, tail, wrap) \ ((head) >= (tail) ? (head) - (tail) : (wrap) - (tail) + (head)) @@ -644,19 +647,19 @@ static int iflib_timer_default = 1000; SYSCTL_INT(_net_iflib, OID_AUTO, timer_default, CTLFLAG_RW, &iflib_timer_default, 0, "number of ticks between iflib_timer calls"); /* - * Consecutive timer periods a TX queue must stay frozen - see - * iflib_timer(), which defines that state - before the hardware is - * asked whether it has completions pending. Four periods is roughly - * two seconds with the default timer interval: a healthy queue on - * hardware that coalesces completion reports (e.g. 8254x, - * TXDCTL.WTHRESH) stays frozen for at most two (measured on 82541PI), - * a wedged one until it is reset. + * Consecutive timer periods a TX queue must stay frozen while demand + * persists - see iflib_timer(), which defines those states - before the + * hardware is asked whether it has completions pending. Four periods is + * roughly two seconds with the default timer interval: a healthy queue on + * hardware that coalesces completion reports (e.g. 8254x, TXDCTL.WTHRESH) + * stays frozen for at most two (measured on 82541PI), a wedged one until it + * is reset. */ static int iflib_tx_watchdog_periods = 4; SYSCTL_INT(_net_iflib, OID_AUTO, tx_watchdog_periods, CTLFLAG_RWTUN, &iflib_tx_watchdog_periods, 0, - "consecutive frozen timer periods before a TX queue is checked for " - "a hang (0 disables the check)"); + "consecutive frozen timer periods under demand before a TX queue is " + "checked for a hang (0 disables the check)"); #if IFLIB_DEBUG_COUNTERS @@ -2494,8 +2497,8 @@ iflib_timer(void *arg) * delays the verdict by one timer period. */ if (this_tick - txq->ift_last_timer_tick >= iflib_timer_default) { - qidx_t outstanding; - bool frozen; + qidx_t in_use, outstanding; + bool demand, frozen; txq->ift_last_timer_tick = this_tick; IFDI_TIMER(ctx, txq->ift_id); @@ -2509,7 +2512,8 @@ iflib_timer(void *arg) * reported and must not count (ift_rs_pending * over-counts it by one per packet). */ - outstanding = txq->ift_in_use - + in_use = txq->ift_in_use; + outstanding = in_use - (qidx_t)(txq->ift_processed - txq->ift_cleaned); /* @@ -2519,12 +2523,17 @@ iflib_timer(void *arg) * up, with no pause frames and no pending doorbell * (the laggard check below rings it). * - * Being frozen is not a fault - the hardware may - * defer marking descriptors as completed - * indefinitely, and 8254x hardware does so for a - * quiet queue - therefore the check arms only when a - * frozen queue also takes on new work, and acts only - * once it has stayed frozen for + * Being frozen is not a fault - the hardware may defer + * marking descriptors as completed indefinitely, and + * 8254x hardware does so for a quiet queue. Continue + * arming only while demand persists: the outstanding + * count grows, the software ring is stalled, or the + * hardware ring has reached iflib's backpressure + * threshold. The last condition covers simple-TX, which + * does not use the software ring. This also prevents one + * mixed lockless counter sample from arming a quiet queue + * until the verdict. Act only once it has stayed frozen + * under demand for * net.iflib.tx_watchdog_periods consecutive periods. */ frozen = outstanding > txq->ift_rs_pending && @@ -2532,10 +2541,12 @@ iflib_timer(void *arg) txq->ift_db_pending == 0 && sctx->isc_pause_frames == 0 && ctx->ifc_link_state == LINK_STATE_UP; - if (!frozen) + demand = outstanding > txq->ift_outstanding_prev || + ifmp_ring_is_stalled(txq->ift_br) || + in_use + MAX_TX_DESC(ctx) >= txq->ift_size - txq->ift_pad; + if (!frozen || !demand) txq->ift_wdog_armed = 0; - else if (txq->ift_wdog_armed > 0 || - outstanding > txq->ift_outstanding_prev) { + else { if (txq->ift_wdog_armed < UINT16_MAX) txq->ift_wdog_armed++; } @@ -2544,8 +2555,8 @@ iflib_timer(void *arg) * Frozen long enough: ask the hardware. Completions * ready but unharvested for this long mean the * completion interrupt went missing - kick the - * queue's task. Nothing ready, although the queue - * kept taking on work, means it is hung. + * queue's task. Nothing ready while demand persisted + * means it is hung. */ if (iflib_tx_watchdog_periods > 0 && txq->ift_wdog_armed >= iflib_tx_watchdog_periods) { @@ -3281,9 +3292,6 @@ txq_max_rs_deferred(iflib_txq_t txq) #define QIDX(ctx, m) ((((m)->m_pkthdr.flowid & ctx->ifc_softc_ctx.isc_rss_table_mask) % NTXQSETS(ctx)) + FIRST_QSET(ctx)) #define DESC_RECLAIMABLE(q) ((int)((q)->ift_processed - (q)->ift_cleaned - (q)->ift_ctx->ifc_softc_ctx.isc_tx_nsegments)) -#define MAX_TX_DESC(ctx) MAX((ctx)->ifc_softc_ctx.isc_tx_tso_segments_max, \ - (ctx)->ifc_softc_ctx.isc_tx_nsegments) - static inline bool iflib_txd_db_check(iflib_txq_t txq, int ring) {