From nobody Fri Aug 21 00:44:59 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hR1mX1zPkz6pP7b for ; Fri, 21 Aug 2026 00:45:00 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hR1mW3WNXz45LN for ; Fri, 21 Aug 2026 00:44:59 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1787273099; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=qlCv+fqUGOFA7BAAGgwJ2wLmxQ8a8fGaeCN78ezFgpg=; b=cjM8r0pGroVP7uMtNsAp9Gw8tADx9mUIv7/J+/zmzR25S7ogD4Cf45Q87kJVbN1Ec/1zDr tvPCkYgmia6mF/HIt5bSJMBbyLvulshEFj1OJYu3J0QVoFEHw6YMNbEAS4z32YhOvDyoFV DwFl9nsnuekZDR6Q7n+5pgRrKVuG3WLkOGoF8gWnDjWKkuS2YF9j76K/8HL2DqDb6oGDMl NY/vV6VXxZDyCmzb4rt4tOZGFSFRZahAJ0cCoa31H21MquKl0SuXCLFz9ieJ4GPAiSJMrW L6AkoKuMxkasqjwl5I3sOAkgTIAHal8qZqoV3bAs4y9m1bqoAUSdHDmrHz6vOw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1787273099; a=rsa-sha256; cv=none; b=hKR6gHj30oXtOqC+pAdxvDNQacsC10m9cXQFLaF3IvaNGqW0taS0Kd/fZo/4xG74k36Xpy av8+9FG339BKCXU8rSb1xu+Rv9vyr7lhK+gFk3NteNoUcs3cR1pYl/Ht7QEhmTcwnYRqMB wYvl6JpMB/X/w4sAaA+itFCfdn8iN42N4ge/nEcEyeckPsctOojLwlIwECIi9Y0CDjeC1q k0JpbvEynM2VXR8l3uFHn+eQj+u5kfYSIO+2Fb4OWqE9mB7tbvfqCOWmGDddYLfi9hwJKe GXFxzPe9ksqoQU79BdfbjdROUCy0xoqJFK5yHc+r2J1hfrjGL5mrYl6m76eShA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1787273099; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=qlCv+fqUGOFA7BAAGgwJ2wLmxQ8a8fGaeCN78ezFgpg=; b=DRK67dZjsfoWnnugQDZR2TQinTTOr+URXN229Qvv+ciM5DNRWtyji+IOmNPS1gnOCNODrq itLEk4AANL18V0ClIeC43UGqJWmv3drvZjHgZHFJG7Vfnn/qzH/CaGA13Kt1ZZP+Kl+hN2 jm9CSBDmgx/6BnA/Nksd1an5qyO1l+UYk7kO7HdNoOHf2Ge4qVlmvhpVYU3P3OAdHoApW7 n/3+MjXWI444UkteJLsWPraPOGW/zPAiv/odRAauE+30xUNso1Ql0kstPQV7+xmdIoW0dl WMzWtLcAEaf0DpkjcyCzGZAVd/1Tqs07pUtHnVQbWMDAVKALr/pKo72MEXN4lA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hR1mW2P0jzh7l for ; Fri, 21 Aug 2026 00:44:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 2474c by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Fri, 21 Aug 2026 00:44:59 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kevin Bowling Subject: git: b5c6cea8bcb6 - stable/14 - ixgbe: Quiesce VFs across PF reset List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kbowling X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: b5c6cea8bcb6a6df258f9c034192bd0a3891cb6e Auto-Submitted: auto-generated Date: Fri, 21 Aug 2026 00:44:59 +0000 Message-Id: <6a879f8b.2474c.6a55a31c@gitrepo.freebsd.org> The branch stable/14 has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=b5c6cea8bcb6a6df258f9c034192bd0a3891cb6e commit b5c6cea8bcb6a6df258f9c034192bd0a3891cb6e Author: Kevin Bowling AuthorDate: 2026-08-06 08:21:45 +0000 Commit: Kevin Bowling CommitDate: 2026-08-21 00:44:09 +0000 ixgbe: Quiesce VFs across PF reset Stop VF transmit and receive in hardware, clear PF-side mailbox CTS, and notify active VFs before resetting a PF. A PF reset invalidates VF queue state, so the no-CTS control message makes cooperative VFs discard stale state and renegotiate after the PF returns. The hardware queue gates synchronously prevent further VF DMA. Do not hold the exclusive iflib context lock for a fixed VF-watchdog interval after the reset. Report the PF link transition directly instead of dispatching mailbox work from the stop path, which could otherwise re-enable VF I/O mid-reset. The CTS, PF-control, and VF queue controls follow the reset mechanisms used by DPDK. (cherry picked from commit aea4240ef5834fb4a47f80c659c80f902cb4bb06) --- sys/dev/ixgbe/if_ix.c | 11 +++++++++-- sys/dev/ixgbe/if_sriov.c | 34 ++++++++++++++++++++++++++++++++++ sys/dev/ixgbe/ixgbe_sriov.h | 2 ++ 3 files changed, 45 insertions(+), 2 deletions(-) diff --git a/sys/dev/ixgbe/if_ix.c b/sys/dev/ixgbe/if_ix.c index 7588cb369a10..1909a936c3c1 100644 --- a/sys/dev/ixgbe/if_ix.c +++ b/sys/dev/ixgbe/if_ix.c @@ -4746,8 +4746,10 @@ ixgbe_if_stop(if_ctx_t ctx) INIT_DEBUGOUT("ixgbe_if_stop: begin\n"); - if (sc->feat_en & IXGBE_FEATURE_SRIOV) + if (sc->feat_en & IXGBE_FEATURE_SRIOV) { ixgbe_disable_mdd(hw); + ixgbe_quiesce_vfs(sc); + } ixgbe_reset_hw(hw); hw->adapter_stopped = false; ixgbe_stop_adapter(hw); @@ -4756,7 +4758,12 @@ ixgbe_if_stop(if_ctx_t ctx) /* Update the stack */ sc->link_up = false; - ixgbe_if_update_admin_status(ctx); + if (sc->link_active) { + if (bootverbose) + device_printf(sc->dev, "Link is Down\n"); + iflib_link_state_change(ctx, LINK_STATE_DOWN, 0); + sc->link_active = false; + } /* reprogram the RAR[0] in case user changed it. */ ixgbe_set_rar(&sc->hw, 0, sc->hw.mac.addr, 0, IXGBE_RAH_AV); diff --git a/sys/dev/ixgbe/if_sriov.c b/sys/dev/ixgbe/if_sriov.c index af0027a073c8..8611f2685611 100644 --- a/sys/dev/ixgbe/if_sriov.c +++ b/sys/dev/ixgbe/if_sriov.c @@ -283,6 +283,40 @@ ixgbe_ping_all_vfs(struct ixgbe_softc *sc) } } /* ixgbe_ping_all_vfs */ +/* + * Stop VF DMA before resetting the PF. A PF reset invalidates the VF queue + * state, so allowing an active VF to resume with its old rings can strand + * descriptors in both the VF and PF. Clearing CTS makes a cooperative VF + * renegotiate its state after the PF comes back; it is deliberately separate + * from IXGBE_VF_IO_DISABLED, which records a persistent administrative or + * recovery decision. + */ +void +ixgbe_quiesce_vfs(struct ixgbe_softc *sc) +{ + struct ixgbe_hw *hw; + struct ixgbe_vf *vf; + uint32_t index, mask, vfre, vfte; + int i; + + hw = &sc->hw; + for (i = 0; i < sc->num_vfs; i++) { + vf = &sc->vfs[i]; + if (!(vf->flags & IXGBE_VF_ACTIVE)) + continue; + + vf->flags &= ~IXGBE_VF_CTS; + index = IXGBE_VF_INDEX(vf->pool); + mask = IXGBE_VF_BIT(vf->pool); + vfte = IXGBE_READ_REG(hw, IXGBE_VFTE(index)); + vfre = IXGBE_READ_REG(hw, IXGBE_VFRE(index)); + IXGBE_WRITE_REG(hw, IXGBE_VFTE(index), vfte & ~mask); + IXGBE_WRITE_REG(hw, IXGBE_VFRE(index), vfre & ~mask); + ixgbe_send_vf_msg(hw, vf, IXGBE_PF_CONTROL_MSG); + } + IXGBE_WRITE_FLUSH(hw); +} /* ixgbe_quiesce_vfs */ + static bool ixgbe_pf_owns_vlan(struct ixgbe_softc *sc, uint16_t tag) diff --git a/sys/dev/ixgbe/ixgbe_sriov.h b/sys/dev/ixgbe/ixgbe_sriov.h index dbe56ec1171c..de0485f6e9a2 100644 --- a/sys/dev/ixgbe/ixgbe_sriov.h +++ b/sys/dev/ixgbe/ixgbe_sriov.h @@ -89,6 +89,7 @@ int ixgbe_iov_validate(struct ixgbe_softc *, u16); void ixgbe_if_iov_uninit(if_ctx_t); void ixgbe_initialize_iov(struct ixgbe_softc *); void ixgbe_activate_vfs(struct ixgbe_softc *); +void ixgbe_quiesce_vfs(struct ixgbe_softc *); void ixgbe_recalculate_max_frame(struct ixgbe_softc *); void ixgbe_ping_all_vfs(struct ixgbe_softc *); void ixgbe_init_iov_recovery(struct ixgbe_softc *); @@ -109,6 +110,7 @@ u32 ixgbe_get_mrqc(int); #define ixgbe_uninit_iov(_a) #define ixgbe_initialize_iov(_a) #define ixgbe_activate_vfs(_a) +#define ixgbe_quiesce_vfs(_a) #define ixgbe_recalculate_max_frame(_a) #define ixgbe_ping_all_vfs(_a) #define ixgbe_init_iov_recovery(_a)