From nobody Mon Aug 24 09:54:14 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hT5pt1x3Gz6p9H0 for ; Mon, 24 Aug 2026 09:54:14 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hT5pt1MVkz47B4 for ; Mon, 24 Aug 2026 09:54:14 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1787565254; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=56FkOqFOKhG5BxGg4GvlzvwLaDvD8hUGzpYyu2VqneY=; b=iKurS/Mns8D+6cve5J19kVCuMO9x3qi8LMUKLRfcZdU+fQs5w/bf9RPwZfMZSbfcQQqkF6 kwkWmij6O8emAUjmItSRXaqBg0Oqbe80ew3g5VEXw/xPAbrT8DufC5T1WqXn4HVjzAinZ5 QEIHsXXxZgRNqddz8GgFK93XWMXOp+t6B96e4oe4/cHuTt4g+aB4dv1gBOvd8B5OQgfsQ/ 9g5nHrX0g4PlXc7QKtLECmRLyldiJpa0slUHzZ5PKBY7Soif8Q71NTRBjra1tyxW+rnC6p bcuilSD6ZxeHtr6BqZu27kABjLldLbTeVKVBHdxUw18m3iV3+ubjqVxYodyPxQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1787565254; a=rsa-sha256; cv=none; b=n8aIlN/iNuEMOSWCMFSrLDQ2UuCzu681ytPdlthd87yvnlOchQwmyrzlkzVQ1d6ztsS72N c/1T9Iw6QuoP/JzrL+3ceKGlqkzTb8i1d6tdLgxZwjG+PSCr92NYLeyhQv5e3niYrb4P3B J59iAkfVaGLwA4uMA+YQtpQEqV2AlefohVLXOZJx+0dYkUPPeDHQBbAdyP4DyHOV7Llcei eu1I1RVaGenKufJmSIl8NH2HXjL6I14Ts5Y0qXhcZMBRiTY/Vh0c/TTLayq7zJDkwl+hEL 7d1D9v09ufkc6tUut4Vb4l4AHlDx+XEYbCAnR4yjcE8QX/ZYLSDFdM05BuIiJQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1787565254; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=56FkOqFOKhG5BxGg4GvlzvwLaDvD8hUGzpYyu2VqneY=; b=wo0AyTPDyIfQrkJ7jk2zkwYfZBI5lorLdTl+oxaQWzuwR0flC4X0LkpWDhM4ZmM5Ha4sbd GCZwrd9C16/cqWOfzAb4uSa41o7AynFcPKTKPYe06R/u7NAb1zitqsnGB33BKo98s4cWYz m8tvOmjxBl2jt9vbe1llx3gUFNuHfcE0kXIz9gGsQlbFiaoGAmqsirGHQ+8dmY+1l3Qf0G krfiDqrehQG0s6G7vDqY5DoYPQSlJBWn3+KDMiZCJ8Wg0Y2wiNSdNKgdN+JY4Jdi+dLpjY Gqw6vT8XJ+CPrYstvMgL6yzj1VXnFUygvbkXtVOSsHWwx8JBM655kuUl3St2ow== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hT5pt0BTHz108X for ; Mon, 24 Aug 2026 09:54:14 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 1ecb2 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Mon, 24 Aug 2026 09:54:14 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Kevin Bowling Subject: git: e7aa5a5a3f69 - main - e1000: Serialize 82579 CSR writes with the Management Engine List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kbowling X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: e7aa5a5a3f690f49488f89e01444ba1bcebc427b Auto-Submitted: auto-generated Date: Mon, 24 Aug 2026 09:54:14 +0000 Message-Id: <6a8c14c6.1ecb2.1357ccaf@gitrepo.freebsd.org> The branch main has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=e7aa5a5a3f690f49488f89e01444ba1bcebc427b commit e7aa5a5a3f690f49488f89e01444ba1bcebc427b Author: Kevin Bowling AuthorDate: 2026-08-16 07:10:09 +0000 Commit: Kevin Bowling CommitDate: 2026-08-24 09:51:38 +0000 e1000: Serialize 82579 CSR writes with the Management Engine The 82579 PCIm2PCI arbiter can acknowledge a host MAC CSR write while the Management Engine is accessing another CSR. The host write can be lost; subsequent target accesses may no longer be claimed by the MAC and can hang the system. For 82579 controllers with valid management firmware, wait for the ME CSR access indication before every MAC CSR write. Keep the wait bounded and use DELAY because writes occur in interrupt and datapath contexts. Verify every transmit and receive tail write. If a tail does not hold the requested value, disable its datapath direction and request a full iflib reset. Keep the ordinary register-write path as a direct MMIO write behind a predicted per-device gate. Contain the wait and tail recovery in the 82579 slow path rather than adding tail-specific accessors and state to the rest of the e1000 family. Documentation on the PCH NICs is scare so Intel's Linux e1000e fixes publicly document the hardware failure and required serialization as commits bdc125f73f3c and d601afcae2fe. This implementation is a bit cleaner. Tested on a Thinkpad T430 (82579LM) with a test kernel to simulate ME contention without incident as well as lost tail writes causing a succesful recovery. MFC after: 2 weeks Sponsored by: BBOX.io --- sys/dev/e1000/e1000_osdep.c | 60 +++++++++++++++++++++++++++++++++++++++++++++ sys/dev/e1000/e1000_osdep.h | 20 +++++++++------ sys/dev/e1000/if_em.c | 9 +++++++ 3 files changed, 82 insertions(+), 7 deletions(-) diff --git a/sys/dev/e1000/e1000_osdep.c b/sys/dev/e1000/e1000_osdep.c index 8b598f18cf12..c38e963f2ea3 100644 --- a/sys/dev/e1000/e1000_osdep.c +++ b/sys/dev/e1000/e1000_osdep.c @@ -36,6 +36,66 @@ int e1000_use_pause_delay = 0; +/* + * Wait while the 82579 Management Engine owns the PCIm2PCI arbiter. DELAY + * is required because CSR writes also occur from interrupt and datapath + * contexts where sleeping is forbidden. + */ +static void +e1000_pcim2pci_arbiter_wait(struct e1000_osdep *osdep) +{ + int i; + + i = E1000_ICH_FWSM_PCIM2PCI_COUNT; + while ((bus_space_read_4(osdep->mem_bus_space_tag, + osdep->mem_bus_space_handle, E1000_FWSM) & + E1000_ICH_FWSM_PCIM2PCI) != 0 && --i != 0) + DELAY(50); +} + +/* + * Serialize an 82579 MAC CSR write against the Management Engine. The + * FreeBSD driver exposes one queue on this controller, so recognize its two + * tail registers here and verify them without imposing tail-specific APIs on + * the rest of the e1000 family. + */ +void +e1000_pcim2pci_write(struct e1000_osdep *osdep, uint32_t reg, uint32_t value) +{ + uint32_t control, control_reg, enable; + const char *direction; + + e1000_pcim2pci_arbiter_wait(osdep); + bus_space_write_4(osdep->mem_bus_space_tag, + osdep->mem_bus_space_handle, reg, value); + + if (reg == E1000_TDT(0)) { + control_reg = E1000_TCTL; + enable = E1000_TCTL_EN; + direction = "transmit"; + } else if (reg == E1000_RDT(0)) { + control_reg = E1000_RCTL; + enable = E1000_RCTL_EN; + direction = "receive"; + } else { + return; + } + if (bus_space_read_4(osdep->mem_bus_space_tag, + osdep->mem_bus_space_handle, reg) == value) + return; + + control = bus_space_read_4(osdep->mem_bus_space_tag, + osdep->mem_bus_space_handle, control_reg); + e1000_pcim2pci_arbiter_wait(osdep); + bus_space_write_4(osdep->mem_bus_space_tag, + osdep->mem_bus_space_handle, control_reg, control & ~enable); + device_printf(osdep->dev, + "Management Engine caused an invalid %s tail write; " + "requesting reset\n", direction); + iflib_request_reset(osdep->ctx); + iflib_admin_intr_deferred(osdep->ctx); +} + static void e1000_enable_pause_delay(void *use_pause_delay) { diff --git a/sys/dev/e1000/e1000_osdep.h b/sys/dev/e1000/e1000_osdep.h index 1226c0264333..b3b16b102edb 100644 --- a/sys/dev/e1000/e1000_osdep.h +++ b/sys/dev/e1000/e1000_osdep.h @@ -161,10 +161,13 @@ struct e1000_osdep bus_space_handle_t flash_bus_space_handle; device_t dev; if_ctx_t ctx; + bool pcim2pci_arbiter_wa; bool vf; bool vf_82576; }; +void e1000_pcim2pci_write(struct e1000_osdep *, uint32_t, uint32_t); + #ifdef INVARIANTS /* * 82576 and I350 VFs expose a sparse register file. Keep this list local to @@ -276,11 +279,6 @@ e1000_vf_reg_valid(uint32_t reg, bool write, bool vf_82576) bus_space_read_4(((struct e1000_osdep *)(hw)->back)->mem_bus_space_tag, \ ((struct e1000_osdep *)(hw)->back)->mem_bus_space_handle, offset) -/* Write to an absolute offset in the adapter's memory space */ -#define E1000_WRITE_OFFSET(hw, offset, value) \ - bus_space_write_4(((struct e1000_osdep *)(hw)->back)->mem_bus_space_tag, \ - ((struct e1000_osdep *)(hw)->back)->mem_bus_space_handle, offset, value) - static __inline uint32_t e1000_rd32(struct e1000_osdep *osdep, uint32_t reg) { @@ -312,10 +310,18 @@ e1000_wr32(struct e1000_osdep *osdep, uint32_t reg, uint32_t value) ("e1000: invalid VF register write at %#x", reg)); #endif - bus_space_write_4(osdep->mem_bus_space_tag, - osdep->mem_bus_space_handle, reg, value); + if (__predict_true(!osdep->pcim2pci_arbiter_wa)) { + bus_space_write_4(osdep->mem_bus_space_tag, + osdep->mem_bus_space_handle, reg, value); + return; + } + e1000_pcim2pci_write(osdep, reg, value); } +/* Write to an absolute offset in the adapter's memory space. */ +#define E1000_WRITE_OFFSET(hw, offset, value) \ + e1000_wr32((hw)->back, (offset), (value)) + /* Register READ/WRITE macros */ #define E1000_READ_REG(hw, reg) \ diff --git a/sys/dev/e1000/if_em.c b/sys/dev/e1000/if_em.c index aeb38fd4fb10..082549767315 100644 --- a/sys/dev/e1000/if_em.c +++ b/sys/dev/e1000/if_em.c @@ -1413,6 +1413,15 @@ em_if_attach_pre(if_ctx_t ctx) error = ENXIO; goto err_pci; } + /* + * 82579 can lose a host CSR write while the Management Engine owns + * the PCIm2PCI arbiter. Enable the OS register write interlock before + * shared code initialization performs any MAC writes. + */ + if (hw->mac.type == e1000_pch2lan && + (E1000_READ_REG(hw, E1000_FWSM) & + E1000_ICH_FWSM_FW_VALID) != 0) + sc->osdep.pcim2pci_arbiter_wa = true; /* ** For ICH8 and family we need to