From nobody Fri Aug 21 00:33:47 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hR1Wb5g4Xz6pNW7 for ; Fri, 21 Aug 2026 00:33:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hR1Wb4xjcz42Nf for ; Fri, 21 Aug 2026 00:33:47 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1787272427; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=SZXvKG5roD/xzWhVtJOoMoaPGWJ05qhKEfs3OhLpE80=; b=FWGz52QYRN0qwtGjZsZM2kfrITf/516b5DN5iNzeBeO7PUyoznvfRUiEcwtnNQ/9GitlbF t9UlAVom76tC59mBceG6klYU6SizWGGZI+CQL6geo7cyRp6WFPmcAPUgZK1oT/GCD8LcEM arkiu9+QPemvRfNeH0Dvi9SpjVu/nZt54bo8xbyigk2FqL3UznhftbJk7Qez2+7I6z1VgI KV/rHnTB8DwxjxAVVzhCwqkplbehUHyiVdLWU+caCGdE1UhgAwWXesn75pO8N6cYZamltO UcLC25j7dfNaLh31G8t66t9XaY/aJJjmQQ5y/YjPLcn5qK1r7jBcHQspv6LCRQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1787272427; a=rsa-sha256; cv=none; b=Rfon5IROWLSsQ7Zystd+sl3a0SoDZ9tRM0f3woVbJylalh+Ee+icTvlF4PJyB8i/NqCxAI 1E2/46tHcwE2X6F+uMjC8M6SONWmstKka96Tm5vy9fCHT9kKIWinxw91Wl8kyqWH734pn8 NA3skalldgkDyV33jKTObDIaol/3y5LHi4qWKaWngVmJ/y0dMQgbsgde9L8Ty/Dm3f/wRE 8/b6NVeElSzG1jqRiELbUMPLTdtyho+sO5MKELVmSGj0J6ZOg0XbGU+J5s6plMhovgsXnE y7EudIpfh8ZVb0RScXH1b2+RF3yKoRTKdMp8Tf0Q5cnbtxCOu7AvR8h+Ygk8gw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1787272427; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=SZXvKG5roD/xzWhVtJOoMoaPGWJ05qhKEfs3OhLpE80=; b=LkapU7u6DS2TCTzXd+JUXl7DDSKL7LK0Qc7hAkldMtCHQed2nwSyTpg6dJrPDurqkg6Eud PLhXfgH2Wrnqn5N4R9KI/qieUWtTNEBKoveMvFas1iy8VSAg/ZcxMz86lYNqUCI4dAJbvh JttL9nTs/U/tMQxdeQzqXS6h+Fd/8jaQHsoTDQGGwSaHd+3sPc6wFAVm3qa+L1lxiB3c9s yBMI7ocfcCUYA96krSjNokqfd4b/IMx6dQOH5iiJAeZvbtfxZ/2R47OQraw/ShLk1XphlE 5ZNtJMCpSRquhG9Hi5Py6d3gdu08Btl8c02BwdAWx00xoL80XKrI+i5q/kfvpQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hR1Wb3GQszh3C for ; Fri, 21 Aug 2026 00:33:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 21fb3 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Fri, 21 Aug 2026 00:33:47 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kevin Bowling Subject: git: 118ab673a947 - stable/15 - ixgbe: Quiesce VFs across PF reset List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kbowling X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: 118ab673a94737c8585f545896b1acfe9da577be Auto-Submitted: auto-generated Date: Fri, 21 Aug 2026 00:33:47 +0000 Message-Id: <6a879ceb.21fb3.df267a@gitrepo.freebsd.org> The branch stable/15 has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=118ab673a94737c8585f545896b1acfe9da577be commit 118ab673a94737c8585f545896b1acfe9da577be Author: Kevin Bowling AuthorDate: 2026-08-06 08:21:45 +0000 Commit: Kevin Bowling CommitDate: 2026-08-21 00:33:38 +0000 ixgbe: Quiesce VFs across PF reset Stop VF transmit and receive in hardware, clear PF-side mailbox CTS, and notify active VFs before resetting a PF. A PF reset invalidates VF queue state, so the no-CTS control message makes cooperative VFs discard stale state and renegotiate after the PF returns. The hardware queue gates synchronously prevent further VF DMA. Do not hold the exclusive iflib context lock for a fixed VF-watchdog interval after the reset. Report the PF link transition directly instead of dispatching mailbox work from the stop path, which could otherwise re-enable VF I/O mid-reset. The CTS, PF-control, and VF queue controls follow the reset mechanisms used by DPDK. (cherry picked from commit aea4240ef5834fb4a47f80c659c80f902cb4bb06) --- sys/dev/ixgbe/if_ix.c | 11 +++++++++-- sys/dev/ixgbe/if_sriov.c | 34 ++++++++++++++++++++++++++++++++++ sys/dev/ixgbe/ixgbe_sriov.h | 2 ++ 3 files changed, 45 insertions(+), 2 deletions(-) diff --git a/sys/dev/ixgbe/if_ix.c b/sys/dev/ixgbe/if_ix.c index 7588cb369a10..1909a936c3c1 100644 --- a/sys/dev/ixgbe/if_ix.c +++ b/sys/dev/ixgbe/if_ix.c @@ -4746,8 +4746,10 @@ ixgbe_if_stop(if_ctx_t ctx) INIT_DEBUGOUT("ixgbe_if_stop: begin\n"); - if (sc->feat_en & IXGBE_FEATURE_SRIOV) + if (sc->feat_en & IXGBE_FEATURE_SRIOV) { ixgbe_disable_mdd(hw); + ixgbe_quiesce_vfs(sc); + } ixgbe_reset_hw(hw); hw->adapter_stopped = false; ixgbe_stop_adapter(hw); @@ -4756,7 +4758,12 @@ ixgbe_if_stop(if_ctx_t ctx) /* Update the stack */ sc->link_up = false; - ixgbe_if_update_admin_status(ctx); + if (sc->link_active) { + if (bootverbose) + device_printf(sc->dev, "Link is Down\n"); + iflib_link_state_change(ctx, LINK_STATE_DOWN, 0); + sc->link_active = false; + } /* reprogram the RAR[0] in case user changed it. */ ixgbe_set_rar(&sc->hw, 0, sc->hw.mac.addr, 0, IXGBE_RAH_AV); diff --git a/sys/dev/ixgbe/if_sriov.c b/sys/dev/ixgbe/if_sriov.c index af0027a073c8..8611f2685611 100644 --- a/sys/dev/ixgbe/if_sriov.c +++ b/sys/dev/ixgbe/if_sriov.c @@ -283,6 +283,40 @@ ixgbe_ping_all_vfs(struct ixgbe_softc *sc) } } /* ixgbe_ping_all_vfs */ +/* + * Stop VF DMA before resetting the PF. A PF reset invalidates the VF queue + * state, so allowing an active VF to resume with its old rings can strand + * descriptors in both the VF and PF. Clearing CTS makes a cooperative VF + * renegotiate its state after the PF comes back; it is deliberately separate + * from IXGBE_VF_IO_DISABLED, which records a persistent administrative or + * recovery decision. + */ +void +ixgbe_quiesce_vfs(struct ixgbe_softc *sc) +{ + struct ixgbe_hw *hw; + struct ixgbe_vf *vf; + uint32_t index, mask, vfre, vfte; + int i; + + hw = &sc->hw; + for (i = 0; i < sc->num_vfs; i++) { + vf = &sc->vfs[i]; + if (!(vf->flags & IXGBE_VF_ACTIVE)) + continue; + + vf->flags &= ~IXGBE_VF_CTS; + index = IXGBE_VF_INDEX(vf->pool); + mask = IXGBE_VF_BIT(vf->pool); + vfte = IXGBE_READ_REG(hw, IXGBE_VFTE(index)); + vfre = IXGBE_READ_REG(hw, IXGBE_VFRE(index)); + IXGBE_WRITE_REG(hw, IXGBE_VFTE(index), vfte & ~mask); + IXGBE_WRITE_REG(hw, IXGBE_VFRE(index), vfre & ~mask); + ixgbe_send_vf_msg(hw, vf, IXGBE_PF_CONTROL_MSG); + } + IXGBE_WRITE_FLUSH(hw); +} /* ixgbe_quiesce_vfs */ + static bool ixgbe_pf_owns_vlan(struct ixgbe_softc *sc, uint16_t tag) diff --git a/sys/dev/ixgbe/ixgbe_sriov.h b/sys/dev/ixgbe/ixgbe_sriov.h index dbe56ec1171c..de0485f6e9a2 100644 --- a/sys/dev/ixgbe/ixgbe_sriov.h +++ b/sys/dev/ixgbe/ixgbe_sriov.h @@ -89,6 +89,7 @@ int ixgbe_iov_validate(struct ixgbe_softc *, u16); void ixgbe_if_iov_uninit(if_ctx_t); void ixgbe_initialize_iov(struct ixgbe_softc *); void ixgbe_activate_vfs(struct ixgbe_softc *); +void ixgbe_quiesce_vfs(struct ixgbe_softc *); void ixgbe_recalculate_max_frame(struct ixgbe_softc *); void ixgbe_ping_all_vfs(struct ixgbe_softc *); void ixgbe_init_iov_recovery(struct ixgbe_softc *); @@ -109,6 +110,7 @@ u32 ixgbe_get_mrqc(int); #define ixgbe_uninit_iov(_a) #define ixgbe_initialize_iov(_a) #define ixgbe_activate_vfs(_a) +#define ixgbe_quiesce_vfs(_a) #define ixgbe_recalculate_max_frame(_a) #define ixgbe_ping_all_vfs(_a) #define ixgbe_init_iov_recovery(_a)