From nobody Mon Aug 24 09:54:14 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hT5pz2XQfz6p8qP for ; Mon, 24 Aug 2026 09:54:19 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hT5pz1xmdz47L6 for ; Mon, 24 Aug 2026 09:54:19 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1787565259; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=56FkOqFOKhG5BxGg4GvlzvwLaDvD8hUGzpYyu2VqneY=; b=adjDclfevEky9REODN4hg9Q3WKZhkql5+2Zx2TAdwxAvGhHzbQDPlIFHzN/mFvEqQrZyWA Lu26bDihbyiaPDNLWZPLXzTYluVIWNKCV9nsERZmYmhSb2enB3JS2JZ9q4Eb4FGOHB5wHw vqiXr9ZNJjGfVLwigEEULO4HNVxuThiXNvaEo4AfwUUVcYbPyRkS6b14ACJQuHrN+JuVNM v7AbX+uq3FCSWQ2Z2l80vprqW9oay7ekxmrwzohr5stL/CD357z3t2jDWabWRwddVQhvT2 X2ahxJK3VQ8wWIyDVqnEHUbi0rZpqLWRGysF9uJ68ez3YQjS+XliFBUbej36hw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1787565259; a=rsa-sha256; cv=none; b=LQnjyj0CprJ9gAi2dDxfcTVduThgOEZN50eH4GySamGdL1VIQcuyp+x+hYGf8dCLM8YOXK MCZ5ZokSE81/y8ySGK8BdnoqdnjFpdydOM8Q60zzdeRv1lb+VcWgwS2TDl2nUtiy8as1d8 WnpiYKjq7Icf7tnCGHz6NEsANEcuKrWr7rjrl/ghxSdJo/j1bzKDQsSKWnsSeBF4utjtwQ BkamZcU1fC6mX3VoVSI6fWxtmRi5Y65J2m1gpttJzZwiObEGAITPWLnYxZ5v+TiY4P/SOu Rbithkk17/O0xZRAAsUH0g5z9qxWiXU4w7AlXtGTTqe7dmqeOuVVU2g9eapNkg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1787565259; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=56FkOqFOKhG5BxGg4GvlzvwLaDvD8hUGzpYyu2VqneY=; b=YKEOoFRwNqHWOUnQwQiDG6xsjACfb8rr7Qy/bcyJtkXH4zUzZlZWn9N04YuaYh8qU23kVH z5CS/32o8dTHww+15l0Nlssg6as/BARNYyMmtn6NlS1td5j5FKj/HNiZ+7F7H3Yz4jEzpW XBNTuesPdNXJC3Cwarp6bnyXzGZMYEPRdUQOoYTYDf6VJZwE38uD8Zyi7hFdXjGgPdMrH7 7P3JO40Hge7x/Sf2ajB6pE6RMzoY28H/B2feyDNZ0L6qIbQ5NvfG4TC2MRMEJjVpFMGJrf 9gkbb9C4Av8ddm2gZG/xXLQqY4/bovOCDzNb6+3OD8CMB4h64fL/N8+kuhmo2Q== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hT5pz0qw4z10FF for ; Mon, 24 Aug 2026 09:54:19 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 1ecb2 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Mon, 24 Aug 2026 09:54:14 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Kevin Bowling Subject: git: e7aa5a5a3f69 - main - e1000: Serialize 82579 CSR writes with the Management Engine List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kbowling X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: e7aa5a5a3f690f49488f89e01444ba1bcebc427b Auto-Submitted: auto-generated Date: Mon, 24 Aug 2026 09:54:14 +0000 Message-Id: <6a8c14c6.1ecb2.1357ccaf@gitrepo.freebsd.org> The branch main has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=e7aa5a5a3f690f49488f89e01444ba1bcebc427b commit e7aa5a5a3f690f49488f89e01444ba1bcebc427b Author: Kevin Bowling AuthorDate: 2026-08-16 07:10:09 +0000 Commit: Kevin Bowling CommitDate: 2026-08-24 09:51:38 +0000 e1000: Serialize 82579 CSR writes with the Management Engine The 82579 PCIm2PCI arbiter can acknowledge a host MAC CSR write while the Management Engine is accessing another CSR. The host write can be lost; subsequent target accesses may no longer be claimed by the MAC and can hang the system. For 82579 controllers with valid management firmware, wait for the ME CSR access indication before every MAC CSR write. Keep the wait bounded and use DELAY because writes occur in interrupt and datapath contexts. Verify every transmit and receive tail write. If a tail does not hold the requested value, disable its datapath direction and request a full iflib reset. Keep the ordinary register-write path as a direct MMIO write behind a predicted per-device gate. Contain the wait and tail recovery in the 82579 slow path rather than adding tail-specific accessors and state to the rest of the e1000 family. Documentation on the PCH NICs is scare so Intel's Linux e1000e fixes publicly document the hardware failure and required serialization as commits bdc125f73f3c and d601afcae2fe. This implementation is a bit cleaner. Tested on a Thinkpad T430 (82579LM) with a test kernel to simulate ME contention without incident as well as lost tail writes causing a succesful recovery. MFC after: 2 weeks Sponsored by: BBOX.io --- sys/dev/e1000/e1000_osdep.c | 60 +++++++++++++++++++++++++++++++++++++++++++++ sys/dev/e1000/e1000_osdep.h | 20 +++++++++------ sys/dev/e1000/if_em.c | 9 +++++++ 3 files changed, 82 insertions(+), 7 deletions(-) diff --git a/sys/dev/e1000/e1000_osdep.c b/sys/dev/e1000/e1000_osdep.c index 8b598f18cf12..c38e963f2ea3 100644 --- a/sys/dev/e1000/e1000_osdep.c +++ b/sys/dev/e1000/e1000_osdep.c @@ -36,6 +36,66 @@ int e1000_use_pause_delay = 0; +/* + * Wait while the 82579 Management Engine owns the PCIm2PCI arbiter. DELAY + * is required because CSR writes also occur from interrupt and datapath + * contexts where sleeping is forbidden. + */ +static void +e1000_pcim2pci_arbiter_wait(struct e1000_osdep *osdep) +{ + int i; + + i = E1000_ICH_FWSM_PCIM2PCI_COUNT; + while ((bus_space_read_4(osdep->mem_bus_space_tag, + osdep->mem_bus_space_handle, E1000_FWSM) & + E1000_ICH_FWSM_PCIM2PCI) != 0 && --i != 0) + DELAY(50); +} + +/* + * Serialize an 82579 MAC CSR write against the Management Engine. The + * FreeBSD driver exposes one queue on this controller, so recognize its two + * tail registers here and verify them without imposing tail-specific APIs on + * the rest of the e1000 family. + */ +void +e1000_pcim2pci_write(struct e1000_osdep *osdep, uint32_t reg, uint32_t value) +{ + uint32_t control, control_reg, enable; + const char *direction; + + e1000_pcim2pci_arbiter_wait(osdep); + bus_space_write_4(osdep->mem_bus_space_tag, + osdep->mem_bus_space_handle, reg, value); + + if (reg == E1000_TDT(0)) { + control_reg = E1000_TCTL; + enable = E1000_TCTL_EN; + direction = "transmit"; + } else if (reg == E1000_RDT(0)) { + control_reg = E1000_RCTL; + enable = E1000_RCTL_EN; + direction = "receive"; + } else { + return; + } + if (bus_space_read_4(osdep->mem_bus_space_tag, + osdep->mem_bus_space_handle, reg) == value) + return; + + control = bus_space_read_4(osdep->mem_bus_space_tag, + osdep->mem_bus_space_handle, control_reg); + e1000_pcim2pci_arbiter_wait(osdep); + bus_space_write_4(osdep->mem_bus_space_tag, + osdep->mem_bus_space_handle, control_reg, control & ~enable); + device_printf(osdep->dev, + "Management Engine caused an invalid %s tail write; " + "requesting reset\n", direction); + iflib_request_reset(osdep->ctx); + iflib_admin_intr_deferred(osdep->ctx); +} + static void e1000_enable_pause_delay(void *use_pause_delay) { diff --git a/sys/dev/e1000/e1000_osdep.h b/sys/dev/e1000/e1000_osdep.h index 1226c0264333..b3b16b102edb 100644 --- a/sys/dev/e1000/e1000_osdep.h +++ b/sys/dev/e1000/e1000_osdep.h @@ -161,10 +161,13 @@ struct e1000_osdep bus_space_handle_t flash_bus_space_handle; device_t dev; if_ctx_t ctx; + bool pcim2pci_arbiter_wa; bool vf; bool vf_82576; }; +void e1000_pcim2pci_write(struct e1000_osdep *, uint32_t, uint32_t); + #ifdef INVARIANTS /* * 82576 and I350 VFs expose a sparse register file. Keep this list local to @@ -276,11 +279,6 @@ e1000_vf_reg_valid(uint32_t reg, bool write, bool vf_82576) bus_space_read_4(((struct e1000_osdep *)(hw)->back)->mem_bus_space_tag, \ ((struct e1000_osdep *)(hw)->back)->mem_bus_space_handle, offset) -/* Write to an absolute offset in the adapter's memory space */ -#define E1000_WRITE_OFFSET(hw, offset, value) \ - bus_space_write_4(((struct e1000_osdep *)(hw)->back)->mem_bus_space_tag, \ - ((struct e1000_osdep *)(hw)->back)->mem_bus_space_handle, offset, value) - static __inline uint32_t e1000_rd32(struct e1000_osdep *osdep, uint32_t reg) { @@ -312,10 +310,18 @@ e1000_wr32(struct e1000_osdep *osdep, uint32_t reg, uint32_t value) ("e1000: invalid VF register write at %#x", reg)); #endif - bus_space_write_4(osdep->mem_bus_space_tag, - osdep->mem_bus_space_handle, reg, value); + if (__predict_true(!osdep->pcim2pci_arbiter_wa)) { + bus_space_write_4(osdep->mem_bus_space_tag, + osdep->mem_bus_space_handle, reg, value); + return; + } + e1000_pcim2pci_write(osdep, reg, value); } +/* Write to an absolute offset in the adapter's memory space. */ +#define E1000_WRITE_OFFSET(hw, offset, value) \ + e1000_wr32((hw)->back, (offset), (value)) + /* Register READ/WRITE macros */ #define E1000_READ_REG(hw, reg) \ diff --git a/sys/dev/e1000/if_em.c b/sys/dev/e1000/if_em.c index aeb38fd4fb10..082549767315 100644 --- a/sys/dev/e1000/if_em.c +++ b/sys/dev/e1000/if_em.c @@ -1413,6 +1413,15 @@ em_if_attach_pre(if_ctx_t ctx) error = ENXIO; goto err_pci; } + /* + * 82579 can lose a host CSR write while the Management Engine owns + * the PCIm2PCI arbiter. Enable the OS register write interlock before + * shared code initialization performs any MAC writes. + */ + if (hw->mac.type == e1000_pch2lan && + (E1000_READ_REG(hw, E1000_FWSM) & + E1000_ICH_FWSM_FW_VALID) != 0) + sc->osdep.pcim2pci_arbiter_wa = true; /* ** For ICH8 and family we need to