From nobody Sat Aug 29 13:23:07 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hXGCj478Hz6qMj0 for ; Sat, 29 Aug 2026 13:23:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hXGCj2BMLz4B3t for ; Sat, 29 Aug 2026 13:23:13 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1788009793; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=8N40LU79OP2fnAfQ/C6j1tGy1uIW3Uq07CfBWaF/vAg=; b=IKZ3Y0L75paV9ylIBKXDxri4SqIiILR4f4zIzCZHGvWUF2ckcbqKdegD01bOOKQazn/uN0 0an47Qv9Z3S09F7Wt/dbsMTWuRtRJw/rKEwA0NjzGORKUA1rUGvaDKqUNUDpDvsTHFJM1u G6zlR8nhljF3VfA+dB8V1opDzDbu5b0XQfeR7PNmPIaOokZJgjW9O1pdHxuZuae7+l1HqH bNgWZzKtlX/5zIIgkrMxN0lxINsOyJSTwomrIR9ShVp9ORZLUpuY6WBA09+WC1Zbj1IeiI 1sCOrufj47RsUBgtmU3orlV9hOl2t/aWF4igjoFBek4am15Ts1tSNYh7U8NbmQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1788009793; a=rsa-sha256; cv=none; b=pGo3nUACJAKOTla9BKy7fZwJFAPt07ZZfctLuQtU+yWGxhEIUf41Bafl1g5vaocpnTlNpf 5tU9GOaLZzj8bhhQtLuq5vVpdhTbsHIoz8ymUk+5VpbQbg19qVEKQc36gVEDAyn0txkjnX RqSXf1tyd0ciI4Z3dB3KJlVsfyU8d9kQqQw4pmZvaLBF8XjyH7AG01ifbMJeEJYk5YMKyL GLkBxOnV93EfIoLpdUJJpf92ARS5arK6cN2dCpFv34oOoJPZ4YogJ2TImT2/PTQ8TMKPaG 4ZZIhFRoaMcV1ziHKmJzBiolSVbqJnWibadNYL2XJ+5O6euVbelHB31JTzAJnA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1788009793; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=8N40LU79OP2fnAfQ/C6j1tGy1uIW3Uq07CfBWaF/vAg=; b=whwUZELfhefpCAuF9e5ZiPMANxEUBVOPHTZIJIT6wHRGd7occuoftb+LSl5auDyJXUvPlY yqDcqPRi2RykZViRG/cbLkmy13noGvFUCHP6/KvPgSHzREZ4uCS/sUq4rIS2DRcHBjSsvL 17WAdCdaOJhrfx+SdEaHCNT2duYzQHQmSDYRBNS/ZfiMXzWpifCJrqla2o4/uUDCVdlaCj 5Q9g8ASV+L67kQwlqNk+fVGvJKxFfTJWB0pDG5hEfKw/jwSC17qCOpa4+MHVfxNAU0ulMz lDKQf+VQmDDE9YpzC58us4ZfeiJYt4TQVTYb/RbL/P9bRoapKIR8pDiD7RZbUg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hXGCj0fb5zhXK for ; Sat, 29 Aug 2026 13:23:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 39927 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Sat, 29 Aug 2026 13:23:07 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Kevin Bowling Subject: git: 245437113020 - main - e1000: Correct igb(4) DMA coalescing register programming List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kbowling X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 245437113020582f3622c9bcc6d841b7d9dc8f96 Auto-Submitted: auto-generated Date: Sat, 29 Aug 2026 13:23:07 +0000 Message-Id: <6a92dd3b.39927.7a9e17de@gitrepo.freebsd.org> The branch main has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=245437113020582f3622c9bcc6d841b7d9dc8f96 commit 245437113020582f3622c9bcc6d841b7d9dc8f96 Author: Kevin Bowling AuthorDate: 2026-08-29 02:38:43 +0000 Commit: Kevin Bowling CommitDate: 2026-08-29 13:23:00 +0000 e1000: Correct igb(4) DMA coalescing register programming The disabled path wrote the complement of DMAC_EN to DMACR. That set every other field, including reserved bits, the one-shot EXIT_DC command, watchdog enables, receive threshold, and PCIe Lx selection. When DMA coalescing was enabled, the requested watchdog and Lx-delay values were ORed into their reset values rather than replacing the fields. PCIEMISC.LX_DECISION was also cleared, preventing DMACR.DMAC_Lx from controlling PCIe low-power entry. Disable coalescing by clearing only DMAC_EN and retaining the documented DMAC_Lx policy and watchdog fields. On enable, replace the variable fields under their masks, select DMA requirements for PCIe low-power entry, and program the loopback and BMC watchdog policies independently of prior state. Make igb_init_dmac() the sole owner of this policy, including when SR-IOV is active, so later IOV initialization cannot overwrite it. Apply this consistently to I350, I354, and I210 while preserving the I354-specific timer units. Leave I210 reserved fields at their required encodings and do not expose the ineffective control on I211. Validated on I210 and I350 hardware. Repeated I210 enable, disable, and reset cycles preserved the watchdog, Lx, TTLX, and LX_DECISION fields. On I350, dmac values of 250, 1000, and 10000 programmed DMACWT as 0x7, 0x1f, and 0x138, respectively, while retaining a four-tick TTLX and leaving the reserved and watchdog fields stable. A real reset initiated by a sibling function restored the enabled dmac=1000 tuple and a down interface path restored the disabled tuple. MFC after: 2 weeks Sponsored by: BBOX.io --- sys/dev/e1000/e1000_defines.h | 2 ++ sys/dev/e1000/if_em.c | 72 ++++++++++++++++++++++++++++--------------- sys/dev/e1000/if_igb_iov.c | 2 -- 3 files changed, 50 insertions(+), 26 deletions(-) diff --git a/sys/dev/e1000/e1000_defines.h b/sys/dev/e1000/e1000_defines.h index 18b311138032..acce4e7042e0 100644 --- a/sys/dev/e1000/e1000_defines.h +++ b/sys/dev/e1000/e1000_defines.h @@ -1546,6 +1546,8 @@ #define E1000_DMACR_DMAC_LX_MASK 0x30000000 #define E1000_DMACR_DMAC_LX_SHIFT 28 #define E1000_DMACR_DMAC_EN 0x80000000 /* Enable DMA Coalescing */ +/* DMA Coalescing VM-to-VM Loopback Watchdog Enable */ +#define E1000_DMACR_DC_LPBKW_EN 0x00004000 /* DMA Coalescing BMC-to-OS Watchdog Enable */ #define E1000_DMACR_DC_BMC2OSW_EN 0x00008000 diff --git a/sys/dev/e1000/if_em.c b/sys/dev/e1000/if_em.c index 649f57762919..7b1ada08e89e 100644 --- a/sys/dev/e1000/if_em.c +++ b/sys/dev/e1000/if_em.c @@ -1208,7 +1208,7 @@ em_add_device_sysctls(struct e1000_softc *sc) CTLTYPE_STRING | CTLFLAG_RD | CTLFLAG_NEEDGIANT, sc, 0, em_get_regs, "A", "Dump Registers"); - if (hw->mac.type >= e1000_i350) { + if (hw->mac.type >= e1000_i350 && hw->mac.type != e1000_i211) { SYSCTL_ADD_PROC(ctx_list, child, OID_AUTO, "dmac", CTLTYPE_INT | CTLFLAG_RW, sc, 0, igb_sysctl_dmac, "I", "DMA Coalesce"); @@ -4268,6 +4268,18 @@ lem_smartspeed(struct e1000_softc *sc) sc->smartspeed = 0; } +static void +igb_disable_dmac(struct e1000_hw *hw) +{ + u32 reg; + + reg = E1000_READ_REG(hw, E1000_DMACR); + reg &= ~E1000_DMACR_DMAC_EN; + /* Retain the documented Lx policy and I210 reserved encoding. */ + reg |= E1000_DMACR_DMAC_LX_MASK; + E1000_WRITE_REG(hw, E1000_DMACR, reg); +} + /********************************************************************* * * Initialize the DMA Coalescing feature @@ -4278,7 +4290,7 @@ igb_init_dmac(struct e1000_softc *sc, u32 pba) { device_t dev = sc->dev; struct e1000_hw *hw = &sc->hw; - u32 dmac, reg = ~E1000_DMACR_DMAC_EN; + u32 dmac, dmacwt, reg, ttlx; u16 hwm; u16 max_frame_size; @@ -4294,7 +4306,7 @@ igb_init_dmac(struct e1000_softc *sc, u32 pba) */ if (igb_iov_enabled(sc)) { if (hw->mac.type > e1000_82580) - E1000_WRITE_REG(hw, E1000_DMACR, 0); + igb_disable_dmac(hw); return; } @@ -4302,7 +4314,7 @@ igb_init_dmac(struct e1000_softc *sc, u32 pba) if (hw->mac.type > e1000_82580) { if (sc->dmac == 0) { /* Disabling it */ - E1000_WRITE_REG(hw, E1000_DMACR, reg); + igb_disable_dmac(hw); return; } else device_printf(dev, "DMA Coalescing enabled\n"); @@ -4324,52 +4336,64 @@ igb_init_dmac(struct e1000_softc *sc, u32 pba) if (dmac < pba - 10) dmac = pba - 10; reg = E1000_READ_REG(hw, E1000_DMACR); - reg &= ~E1000_DMACR_DMACTHR_MASK; + reg &= ~(E1000_DMACR_DMACWT_MASK | + E1000_DMACR_DMACTHR_MASK | E1000_DMACR_DMAC_LX_MASK | + E1000_DMACR_DMAC_EN | E1000_DMACR_DC_LPBKW_EN | + E1000_DMACR_DC_BMC2OSW_EN); reg |= ((dmac << E1000_DMACR_DMACTHR_SHIFT) & E1000_DMACR_DMACTHR_MASK); - /* transition to L0x or L1 if available..*/ + /* Transition to L0s or L1 if available. */ reg |= (E1000_DMACR_DMAC_EN | E1000_DMACR_DMAC_LX_MASK); - /* Check if status is 2.5Gb backplane connection - * before configuration of watchdog timer, which is - * in msec values in 12.8usec intervals - * watchdog timer= msec values in 32usec intervals - * for non 2.5Gb connection - */ + /* + * The watchdog uses 12.8 usec units on an I354 2.5 Gb/s + * backplane connection and 32 usec units otherwise. + */ if (hw->mac.type == e1000_i354) { int status = E1000_READ_REG(hw, E1000_STATUS); if ((status & E1000_STATUS_2P5_SKU) && (!(status & E1000_STATUS_2P5_SKU_OVER))) - reg |= ((sc->dmac * 5) >> 6); + dmacwt = (sc->dmac * 5) >> 6; else - reg |= (sc->dmac >> 5); + dmacwt = sc->dmac >> 5; } else { - reg |= (sc->dmac >> 5); + dmacwt = sc->dmac >> 5; } + reg |= dmacwt & E1000_DMACR_DMACWT_MASK; + if (hw->mac.type == e1000_i350 || + hw->mac.type == e1000_i354) + reg |= E1000_DMACR_DC_LPBKW_EN; + if (hw->mac.type == e1000_i354) + reg |= E1000_DMACR_DC_BMC2OSW_EN; E1000_WRITE_REG(hw, E1000_DMACR, reg); E1000_WRITE_REG(hw, E1000_DMCRTRH, 0); - /* Set the interval before transition */ + /* Set the interval before transition. */ reg = E1000_READ_REG(hw, E1000_DMCTLX); + reg &= ~E1000_DMCTLX_TTLX_MASK; if (hw->mac.type == e1000_i350) reg |= IGB_DMCTLX_DCFLUSH_DIS; /* - ** in 2.5Gb connection, TTLX unit is 0.4 usec - ** which is 0x4*2 = 0xA. But delay is still 4 usec - */ - if (hw->mac.type == e1000_i354) { + * I210 documents TTLX as reserved with a required value of 0x20. + * At 2.5 Gb/s the I354 unit is 0.4 usec, so ten ticks retain + * the four usec interval used at other speeds. + */ + if (hw->mac.type == e1000_i210) { + ttlx = 0x20; + } else if (hw->mac.type == e1000_i354) { int status = E1000_READ_REG(hw, E1000_STATUS); if ((status & E1000_STATUS_2P5_SKU) && (!(status & E1000_STATUS_2P5_SKU_OVER))) - reg |= 0xA; + ttlx = 0xA; else - reg |= 0x4; + ttlx = 0x4; } else { - reg |= 0x4; + ttlx = 0x4; } + reg |= ttlx & E1000_DMCTLX_TTLX_MASK; E1000_WRITE_REG(hw, E1000_DMCTLX, reg); @@ -4379,7 +4403,7 @@ igb_init_dmac(struct e1000_softc *sc, u32 pba) /* make low power state decision controlled by DMA coal */ reg = E1000_READ_REG(hw, E1000_PCIEMISC); - reg &= ~E1000_PCIEMISC_LX_DECISION; + reg |= E1000_PCIEMISC_LX_DECISION; E1000_WRITE_REG(hw, E1000_PCIEMISC, reg); } else if (hw->mac.type == e1000_82580) { diff --git a/sys/dev/e1000/if_igb_iov.c b/sys/dev/e1000/if_igb_iov.c index dc4b23397431..f75ba66d8987 100644 --- a/sys/dev/e1000/if_igb_iov.c +++ b/sys/dev/e1000/if_igb_iov.c @@ -1929,8 +1929,6 @@ igb_iov_initialize(struct e1000_softc *sc) rctl = E1000_READ_REG(hw, E1000_RCTL); E1000_WRITE_REG(hw, E1000_RCTL, rctl | E1000_RCTL_VFE); E1000_WRITE_REG(hw, E1000_MBVFIMR, igb_iov_active_mask(sc)); - if (hw->mac.type == e1000_i350) - E1000_WRITE_REG(hw, E1000_DMACR, 0); ctrl_ext = E1000_READ_REG(hw, E1000_CTRL_EXT); E1000_WRITE_REG(hw, E1000_CTRL_EXT,