git: 46cf612d98e9 - main - e1000: Rework Wake-on-LAN policy and programming

From: Kevin Bowling <kbowling_at_FreeBSD.org>
Date: Mon, 31 Aug 2026 13:04:59 UTC
The branch main has been updated by kbowling:

URL: https://cgit.FreeBSD.org/src/commit/?id=46cf612d98e99967204185829fe829350cfa51a1

commit 46cf612d98e99967204185829fe829350cfa51a1
Author:     Kevin Bowling <kbowling@FreeBSD.org>
AuthorDate: 2026-08-30 06:06:02 +0000
Commit:     Kevin Bowling <kbowling@FreeBSD.org>
CommitDate: 2026-08-31 13:04:28 +0000

    e1000: Rework Wake-on-LAN policy and programming
    
    The driver used the NVM APME default as both the hardware-support
    decision and the mutable filter mask.  Consequently, an NVM-disabled
    but capable port did not advertise wake support, disabling a wake mode
    once could keep it disabled across later suspends, and directed-unicast
    wake could never be selected.
    
    Require the PCI power management capability to report D3hot PME support
    before advertising or arming wake.  A PM capability alone does not mean
    the function can signal PME from the state used during system sleep.
    
    Separate the board and port capability matrix from the NVM-selected
    magic packet default.  Read the proper per function NVM word on igb
    controllers, cover the newer PCH generations, and retain the documented
    legacy, multi-port, and OEM restrictions.  Decode the distinct APM
    Enable locations used by 82544, 82541EI/82547EI, and the later 8254x
    parts.  Do not advertise wake on the 82541ER, whose power-management
    logic cannot assert PME for wake events.  For I210/I211 internal iNVM,
    use the hardware-loaded WUC.APME state; the shared reader does not
    expose the optional Initialization Control 3 word.
    
    Build WUFC from the enabled ifnet capabilities for each suspend.
    Reconstruct RAR0, the multicast table, and the receive filter after the
    stop-time reset so unicast and multicast wake use the current interface
    state.  Fill the MTA on legacy PCI/PCI-X controllers and 82575 through
    82580 when the address list overflows; their multicast wake matchers
    require the indexed MTA bit and do not use RCTL.MPE as a substitute.
    
    Do not access PF-only wake CSRs from the igb VF suspend and resume
    paths.
    
    Use the shared BM page access helpers and propagate every PHY receive
    address and wake-register programming failure.  On resume, perform the
    required LCD reset before clearing host PHY-wake ownership, report the
    saved PHY or MAC wake cause, and clear PCI PME after removing the device
    wake source.  Preserve management engine wake ownership throughout.
    Keep WUC.APME set only when early 82545EM/82546EB manageability needs
    its D3 clock-tree workaround; ordinary host wake uses PCI PME.
    
    Keep the link powered while host wake is armed.  With no host wake,
    evaluate management pass-through at each suspend.  Leave a
    management-owned link untouched and keep PCI PME enabled.  Otherwise,
    use the Intel shared code PHY power-down hook, or its matching SerDes
    shutdown hook on igb fiber and SerDes devices.  Track that state and
    restore the link without another PHY reset before hardware
    initialization.  Ordinary ifconfig down behavior is unchanged.
    
    Apply and undo the PCH Sx workarounds across their full supported range.
    Use controller-specific CTRL and laser semantics, and restore RCTL when
    wake setup fails.  Always run the pending PCIe-transaction drain and
    bus-master-disable sequence before D3.  Suspend reports a
    wake programming failure rather than sleeping without wake, shutdown
    logs it and continues through the fencing sequence.
    
    Do not apply the ICH/PCH IGP3 D3 power-down workaround to igb
    controllers.  The merged driver inherited an unconditional call from the
    em-only driver.  On 82575 and 82576 it asserted CTRL.PHY_RST after the
    wake filters were armed, preventing the link from receiving wake
    traffic.
    
    The implementation was checked against the Intel controller data
    sheets, the Intel Linux e1000, e1000e, and igb lifecycle code, DPDK,
    and the Intel FreeBSD em-7.7.8 and igb-2.5.31 drivers.  The 8254x audit
    also covered the PCI/PCI-X Software Developer's Manual, the 82541/82547
    NVM guide, and the 82544, 82545, and 82546 specification updates.  The
    out of tree drivers carry the family-specific power down and reset block
    hooks but do not call them from suspend.  DPDK supplies the stop/start
    pairing.
    
    On PCH controllers including an 82579LM, I217-LM, and various I219s,
    device-only D3 tests observed PME and BM_WUS.MAG for a magic packet, no
    PME with every host filter disabled, and BM_WUS.EX with only
    directed-unicast wake enabled.  With dev.em.0.wake enabled, ACPI S3
    slept until a delayed magic packet and resumed with the interface
    operational.  After wake traffic stopped and resume completed, a second
    cycle again waited for a newly delayed magic packet.  Traffic restored
    after both host and firmware wake were enabled.
    
    An 82574L woke from S3 after one magic packet, reported MAC wakeup
    status, and returned with link and traffic operational.
    
    On 82571EB and 82573L adapters, device-only D3 tests observed WUS.MAG
    and PCI PME status after a magic packet, then returned to D0 with link
    and traffic operational.  Full S3 did not wake either add-in card.
    The positive device tests and negative S3 isolate the remaining failure
    outside the MAC filter programming and my cards may lack aux power
    wiring because the link was off in S3.
    
    On 82575EB and 82576 adapters, pre-fix device only D3 tests left PMCSR
    at 0x2103 despite ten verified magic packets, and the handoff showed
    CTRL.PHY_RST asserted.  With the em-family gate, identical tests changed
    PMCSR from 0x2103 to 0xa103, resume reported WUS.MAG, and both links
    returned operational.  S3 testing on these separated controller from
    board behavior.  An Intel 82576 card retained link in S3 and woke the
    system from a delayed magic packet, reported WUS.MAG, and returned with
    interface operational.  The tested 82575 add-in card lost its
    link LED in S3 and retained no WUS cause after manual resume, although
    its identical D3hot test passed.   That points the 82575 S3 result to
    card aux power wiring as well.
    
    D3 tests were performed on I210 and I350 but S3 has not yet been
    attempted on them.
    
    lem(4) testing has not been attempted yet.
    
    Community reports of success and failure are welcome.
    
    PR:             232708, 238411, 295443, 296675
    MFC after:      2 weeks
    Sponsored by:   BBOX.io
---
 share/man/man4/em.4           |  26 +-
 sys/dev/e1000/e1000_ich8lan.c |  41 ++-
 sys/dev/e1000/e1000_ich8lan.h |   2 +-
 sys/dev/e1000/e1000_regs.h    |   2 +-
 sys/dev/e1000/if_em.c         | 596 +++++++++++++++++++++++++++++++++---------
 sys/dev/e1000/if_em.h         |  13 +-
 6 files changed, 531 insertions(+), 149 deletions(-)

diff --git a/share/man/man4/em.4 b/share/man/man4/em.4
index 438812a4600f..512f0dc11dc9 100644
--- a/share/man/man4/em.4
+++ b/share/man/man4/em.4
@@ -32,7 +32,7 @@
 .\"
 .\" * Other names and brands may be claimed as the property of others.
 .\"
-.Dd August 11, 2026
+.Dd August 29, 2026
 .Dt EM 4
 .Os
 .Sh NAME
@@ -93,6 +93,30 @@ interfaces.
 The driver supports Transmit/Receive checksum offload and Jumbo Frames
 on all but 82542-based adapters.
 .Pp
+Physical functions on supported boards and ports advertise one or more of the
+.Cm wol_magic ,
+.Cm wol_ucast ,
+and
+.Cm wol_mcast
+capabilities shown by
+.Xr ifconfig 8 .
+The available modes depend on the controller and board.
+The NVM APME setting normally selects whether magic-packet wake is enabled by
+default without hiding modes that the hardware and board support.
+Wake support can be restricted to particular ports on multi-port adapters and
+is not available on
+.Nm igbv
+virtual functions.
+Waking the system from an ACPI sleep state also requires enabling the
+corresponding device wake source when it is exposed, for example:
+.Bd -literal -offset indent
+sysctl dev.em.0.wake=1
+.Ed
+The
+.Xr ifconfig 8
+wake capabilities select the controller packet filters; they do not enable
+the ACPI wake source.
+.Pp
 Furthermore it supports TCP segmentation offload (TSO) on all adapters but
 those based on the 82542, 82543, 82544 and 82547 controller chips.
 The identification LEDs of the adapters supported by the
diff --git a/sys/dev/e1000/e1000_ich8lan.c b/sys/dev/e1000/e1000_ich8lan.c
index eb6971b0e530..ac3ab11c01cb 100644
--- a/sys/dev/e1000/e1000_ich8lan.c
+++ b/sys/dev/e1000/e1000_ich8lan.c
@@ -2815,18 +2815,20 @@ release:
 /**
  *  e1000_copy_rx_addrs_to_phy_ich8lan - Copy Rx addresses from MAC to PHY
  *  @hw:   pointer to the HW structure
+ *
+ *  Returns E1000_SUCCESS when every PHY receive address was programmed.
  **/
-void e1000_copy_rx_addrs_to_phy_ich8lan(struct e1000_hw *hw)
+s32 e1000_copy_rx_addrs_to_phy_ich8lan(struct e1000_hw *hw)
 {
 	u32 mac_reg;
 	u16 i, phy_reg = 0;
-	s32 ret_val;
+	s32 ret_val, restore_val;
 
 	DEBUGFUNC("e1000_copy_rx_addrs_to_phy_ich8lan");
 
 	ret_val = hw->phy.ops.acquire(hw);
 	if (ret_val)
-		return;
+		return ret_val;
 	ret_val = e1000_enable_phy_wakeup_reg_access_bm(hw, &phy_reg);
 	if (ret_val)
 		goto release;
@@ -2834,23 +2836,34 @@ void e1000_copy_rx_addrs_to_phy_ich8lan(struct e1000_hw *hw)
 	/* Copy both RAL/H (rar_entry_count) and SHRAL/H to PHY */
 	for (i = 0; i < (hw->mac.rar_entry_count); i++) {
 		mac_reg = E1000_READ_REG(hw, E1000_RAL(i));
-		hw->phy.ops.write_reg_page(hw, BM_RAR_L(i),
-					   (u16)(mac_reg & 0xFFFF));
-		hw->phy.ops.write_reg_page(hw, BM_RAR_M(i),
-					   (u16)((mac_reg >> 16) & 0xFFFF));
+		ret_val = hw->phy.ops.write_reg_page(hw, BM_RAR_L(i),
+		    (u16)(mac_reg & 0xFFFF));
+		if (ret_val)
+			goto restore;
+		ret_val = hw->phy.ops.write_reg_page(hw, BM_RAR_M(i),
+		    (u16)((mac_reg >> 16) & 0xFFFF));
+		if (ret_val)
+			goto restore;
 
 		mac_reg = E1000_READ_REG(hw, E1000_RAH(i));
-		hw->phy.ops.write_reg_page(hw, BM_RAR_H(i),
-					   (u16)(mac_reg & 0xFFFF));
-		hw->phy.ops.write_reg_page(hw, BM_RAR_CTRL(i),
-					   (u16)((mac_reg & E1000_RAH_AV)
-						 >> 16));
+		ret_val = hw->phy.ops.write_reg_page(hw, BM_RAR_H(i),
+		    (u16)(mac_reg & 0xFFFF));
+		if (ret_val)
+			goto restore;
+		ret_val = hw->phy.ops.write_reg_page(hw, BM_RAR_CTRL(i),
+		    (u16)((mac_reg & E1000_RAH_AV) >> 16));
+		if (ret_val)
+			goto restore;
 	}
 
-	e1000_disable_phy_wakeup_reg_access_bm(hw, &phy_reg);
+restore:
+	restore_val = e1000_disable_phy_wakeup_reg_access_bm(hw, &phy_reg);
+	if (ret_val == E1000_SUCCESS)
+		ret_val = restore_val;
 
 release:
 	hw->phy.ops.release(hw);
+	return ret_val;
 }
 
 static u32 e1000_calc_rx_da_crc(u8 mac[])
@@ -2920,7 +2933,7 @@ s32 e1000_lv_jumbo_workaround_ich8lan(struct e1000_hw *hw, bool enable)
 		}
 
 		/* Write Rx addresses to the PHY */
-		e1000_copy_rx_addrs_to_phy_ich8lan(hw);
+		(void)e1000_copy_rx_addrs_to_phy_ich8lan(hw);
 
 		/* Enable jumbo frame workaround in the MAC */
 		mac_reg = E1000_READ_REG(hw, E1000_FFLT_DBG);
diff --git a/sys/dev/e1000/e1000_ich8lan.h b/sys/dev/e1000/e1000_ich8lan.h
index f3e87f24c9f6..f24170a19285 100644
--- a/sys/dev/e1000/e1000_ich8lan.h
+++ b/sys/dev/e1000/e1000_ich8lan.h
@@ -342,7 +342,7 @@ void e1000_suspend_workarounds_ich8lan(struct e1000_hw *hw);
 u32 e1000_resume_workarounds_pchlan(struct e1000_hw *hw);
 s32 e1000_configure_k1_ich8lan(struct e1000_hw *hw, bool k1_enable);
 s32 e1000_configure_k0s_lpt(struct e1000_hw *hw, u8 entry_latency, u8 min_time);
-void e1000_copy_rx_addrs_to_phy_ich8lan(struct e1000_hw *hw);
+s32 e1000_copy_rx_addrs_to_phy_ich8lan(struct e1000_hw *hw);
 s32 e1000_lv_jumbo_workaround_ich8lan(struct e1000_hw *hw, bool enable);
 s32 e1000_read_emi_reg_locked(struct e1000_hw *hw, u16 addr, u16 *data);
 s32 e1000_write_emi_reg_locked(struct e1000_hw *hw, u16 addr, u16 data);
diff --git a/sys/dev/e1000/e1000_regs.h b/sys/dev/e1000/e1000_regs.h
index a2c724247014..b70fc4aa1255 100644
--- a/sys/dev/e1000/e1000_regs.h
+++ b/sys/dev/e1000/e1000_regs.h
@@ -517,7 +517,7 @@
 #define E1000_VFQA1	0x0B200  /* VLAN Filter Queue Array 1 - RW Array */
 #define E1000_WUC	0x05800  /* Wakeup Control - RW */
 #define E1000_WUFC	0x05808  /* Wakeup Filter Control - RW */
-#define E1000_WUS	0x05810  /* Wakeup Status - RO */
+#define E1000_WUS	0x05810  /* Wakeup Status - RW1C */
 /* Management registers */
 #define E1000_MANC	0x05820  /* Management Control - RW */
 #define E1000_IPAV	0x05838  /* IP Address Valid - RW */
diff --git a/sys/dev/e1000/if_em.c b/sys/dev/e1000/if_em.c
index 3acc4b03f34c..19717d9e5371 100644
--- a/sys/dev/e1000/if_em.c
+++ b/sys/dev/e1000/if_em.c
@@ -507,8 +507,12 @@ static void	em_release_manageability(struct e1000_softc *);
 static void	em_get_hw_control(struct e1000_softc *);
 static void	em_release_hw_control(struct e1000_softc *);
 static void	em_get_wakeup(if_ctx_t);
-static void	em_enable_wakeup(if_ctx_t);
-static int	em_enable_phy_wakeup(struct e1000_softc *);
+static void	em_fill_wakeup_mta(struct e1000_hw *);
+static int	em_enable_wakeup(if_ctx_t);
+static int	em_enable_phy_wakeup(struct e1000_softc *, u32);
+static int	em_disable_phy_wakeup(struct e1000_softc *, u16 *);
+static void	em_power_up_wakeup_link(struct e1000_softc *);
+static void	em_power_down_wakeup_link(struct e1000_softc *);
 static void	em_disable_aspm(struct e1000_softc *);
 
 int		em_intr(void *);
@@ -1651,11 +1655,6 @@ em_if_attach_pre(if_ctx_t ctx)
 	 */
 	if (!sc->vf_ifp) {
 		em_get_wakeup(ctx);
-
-		/* Enable only WOL MAGIC by default. */
-		scctx->isc_capenable &= ~IFCAP_WOL;
-		if (sc->wol != 0)
-			scctx->isc_capenable |= IFCAP_WOL_MAGIC;
 	}
 
 	iflib_set_mac(ctx, hw->mac.addr);
@@ -1766,7 +1765,13 @@ em_if_detach(if_ctx_t ctx)
 static int
 em_if_shutdown(if_ctx_t ctx)
 {
-	return em_if_suspend(ctx);
+	int error;
+
+	error = em_if_suspend(ctx);
+	if (error != 0)
+		device_printf(iflib_get_dev(ctx),
+		    "Wake configuration failed during shutdown: %d\n", error);
+	return (0);
 }
 
 /*
@@ -1776,26 +1781,57 @@ static int
 em_if_suspend(if_ctx_t ctx)
 {
 	struct e1000_softc *sc = iflib_get_softc(ctx);
+	int error;
 
 	if (sc->vf_ifp) {
 		igbv_queue_retry_stop(sc);
 		igbv_mbx_retry_stop(sc);
 	}
+	error = em_enable_wakeup(ctx);
 	em_release_manageability(sc);
 	em_release_hw_control(sc);
-	em_enable_wakeup(ctx);
-	return (0);
+	return (error);
 }
 
 static int
 em_if_resume(if_ctx_t ctx)
 {
 	struct e1000_softc *sc = iflib_get_softc(ctx);
+	u32 wus;
+	u16 phy_wus;
+	int error;
 
-	if (sc->hw.mac.type == e1000_pch2lan)
+	if (sc->hw.mac.type >= e1000_pch2lan &&
+	    sc->hw.mac.type < igb_mac_min)
 		e1000_resume_workarounds_pchlan(&sc->hw);
 
-	return(0);
+	if (sc->wol_phy_armed) {
+		/*
+		 * The PHY wake sequence requires an LCD reset before host wake
+		 * ownership is cleared.  Wake registers survive this reset.
+		 */
+		(void)e1000_phy_hw_reset(&sc->hw);
+		error = em_disable_phy_wakeup(sc, &phy_wus);
+		if (error != E1000_SUCCESS)
+			device_printf(sc->dev,
+			    "Could not clear PHY wakeup state: %d\n", error);
+		else if (phy_wus != 0)
+			device_printf(sc->dev, "PHY wakeup status: %#06x\n",
+			    phy_wus);
+	}
+	if (!sc->vf_ifp && sc->hw.mac.type >= e1000_82544) {
+		wus = E1000_READ_REG(&sc->hw, E1000_WUS);
+		if (!sc->wol_phy_wakeup && wus != 0)
+			device_printf(sc->dev, "MAC wakeup status: %#010x\n",
+			    wus);
+		E1000_WRITE_REG(&sc->hw, E1000_WUFC, 0);
+		E1000_WRITE_REG(&sc->hw, E1000_WUC, 0);
+		E1000_WRITE_REG(&sc->hw, E1000_WUS, ~0U);
+	}
+	/* Clear PME after its MAC or PHY wake source has been removed. */
+	pci_clear_pme(sc->dev);
+
+	return (0);
 }
 
 static int
@@ -1874,6 +1910,8 @@ em_if_init(if_ctx_t ctx)
 		igbv_mbx_retry_prepare(sc);
 		sc->vf_reset_pending = true;
 	}
+	if (sc->suspend_link_powered_down)
+		em_power_up_wakeup_link(sc);
 
 	/* Get the latest mac address, User can use a LAA */
 	bcopy(if_getlladdr(ifp), sc->hw.mac.addr, ETHER_ADDR_LEN);
@@ -3699,6 +3737,19 @@ em_copy_maddr(void *arg, struct sockaddr_dl *sdl, u_int idx)
 	return (1);
 }
 
+/* Make every multicast hash eligible on parts whose wake matcher needs MTA. */
+static void
+em_fill_wakeup_mta(struct e1000_hw *hw)
+{
+	int i;
+
+	memset(hw->mac.mta_shadow, 0xff, sizeof(hw->mac.mta_shadow));
+	for (i = hw->mac.mta_reg_count - 1; i >= 0; i--)
+		E1000_WRITE_REG_ARRAY(hw, E1000_MTA, i,
+		    hw->mac.mta_shadow[i]);
+	E1000_WRITE_FLUSH(hw);
+}
+
 /*********************************************************************
  *  Multicast Update
  *
@@ -6479,29 +6530,41 @@ static void
 em_get_wakeup(if_ctx_t ctx)
 {
 	struct e1000_softc *sc = iflib_get_softc(ctx);
+	if_softc_ctx_t scctx = iflib_get_softc_ctx(ctx);
 	device_t dev = iflib_get_dev(ctx);
 	u16 eeprom_data = 0, device_id, apme_mask;
+	bool apme;
+	int error, wol_capabilities;
 
 	sc->has_manage = e1000_enable_mng_pass_thru(&sc->hw);
-	apme_mask = EM_EEPROM_APME;
+	apme_mask = EM_EEPROM_APME_HIGH;
+	error = E1000_SUCCESS;
 
 	switch (sc->hw.mac.type) {
 	case e1000_82542:
 	case e1000_82543:
 		break;
 	case e1000_82544:
-		e1000_read_nvm(&sc->hw,
+		error = e1000_read_nvm(&sc->hw,
 		    NVM_INIT_CONTROL2_REG, 1, &eeprom_data);
-		apme_mask = EM_82544_APME;
+		apme_mask = EM_EEPROM_APME_LOW;
+		break;
+	case e1000_82541:
+	case e1000_82547:
+		error = e1000_read_nvm(&sc->hw,
+		    NVM_INIT_CONTROL3_PORT_A, 1, &eeprom_data);
+		/* The EI parts place APM Enable in the low byte. */
+		if (sc->hw.device_id != E1000_DEV_ID_82541ER_LOM)
+			apme_mask = EM_EEPROM_APME_LOW;
 		break;
 	case e1000_82546:
 	case e1000_82546_rev_3:
 		if (sc->hw.bus.func == 1) {
-			e1000_read_nvm(&sc->hw,
+			error = e1000_read_nvm(&sc->hw,
 			    NVM_INIT_CONTROL3_PORT_B, 1, &eeprom_data);
 			break;
 		} else
-			e1000_read_nvm(&sc->hw,
+			error = e1000_read_nvm(&sc->hw,
 			    NVM_INIT_CONTROL3_PORT_A, 1, &eeprom_data);
 		break;
 	case e1000_82573:
@@ -6512,11 +6575,11 @@ em_get_wakeup(if_ctx_t ctx)
 	case e1000_82572:
 	case e1000_80003es2lan:
 		if (sc->hw.bus.func == 1) {
-			e1000_read_nvm(&sc->hw,
+			error = e1000_read_nvm(&sc->hw,
 			    NVM_INIT_CONTROL3_PORT_B, 1, &eeprom_data);
 			break;
 		} else
-			e1000_read_nvm(&sc->hw,
+			error = e1000_read_nvm(&sc->hw,
 			    NVM_INIT_CONTROL3_PORT_A, 1, &eeprom_data);
 		break;
 	case e1000_ich8lan:
@@ -6526,172 +6589,406 @@ em_get_wakeup(if_ctx_t ctx)
 	case e1000_pch2lan:
 	case e1000_pch_lpt:
 	case e1000_pch_spt:
-	case e1000_82575:	/* listing all igb devices */
+	case e1000_pch_cnp:
+	case e1000_pch_tgp:
+	case e1000_pch_adp:
+	case e1000_pch_mtp:
+	case e1000_pch_ptp:
+	case e1000_pch_nvp:
+		apme_mask = E1000_WUC_APME;
+		sc->has_amt = true;
+		eeprom_data = E1000_READ_REG(&sc->hw, E1000_WUC);
+		if (sc->hw.mac.type > e1000_ich10lan &&
+		    (eeprom_data & E1000_WUC_PHY_WAKE) != 0)
+			sc->wol_phy_wakeup = true;
+		break;
+	case e1000_82575:
 	case e1000_82576:
+		if (sc->hw.bus.func == 1)
+			error = e1000_read_nvm(&sc->hw,
+			    NVM_INIT_CONTROL3_PORT_B, 1, &eeprom_data);
+		else
+			error = e1000_read_nvm(&sc->hw,
+			    NVM_INIT_CONTROL3_PORT_A, 1, &eeprom_data);
+		sc->has_amt = true;
+		break;
 	case e1000_82580:
 	case e1000_i350:
 	case e1000_i354:
 	case e1000_i210:
 	case e1000_i211:
-		apme_mask = E1000_WUC_APME;
+		error = e1000_read_nvm(&sc->hw,
+		    NVM_INIT_CONTROL3_PORT_A +
+		    NVM_82580_LAN_FUNC_OFFSET(sc->hw.bus.func), 1,
+		    &eeprom_data);
 		sc->has_amt = true;
-		eeprom_data = E1000_READ_REG(&sc->hw, E1000_WUC);
 		break;
 	default:
-		e1000_read_nvm(&sc->hw,
+		error = e1000_read_nvm(&sc->hw,
 		    NVM_INIT_CONTROL3_PORT_A, 1, &eeprom_data);
 		break;
 	}
-	if (eeprom_data & apme_mask)
-		sc->wol = (E1000_WUFC_MAG | E1000_WUFC_MC);
-	/*
-	 * We have the eeprom settings, now apply the special cases
-	 * where the eeprom may be wrong or the board won't support
-	 * wake on lan on a particular port
-	 */
+	if (error != E1000_SUCCESS && bootverbose)
+		device_printf(dev, "NVM read failed while checking WoL: %d\n",
+		    error);
+	if ((sc->hw.mac.type == e1000_i210 ||
+	    sc->hw.mac.type == e1000_i211) &&
+	    sc->hw.nvm.type == e1000_nvm_invm) {
+		/* The shared reader does not expose the optional iNVM word. */
+		apme = (E1000_READ_REG(&sc->hw, E1000_WUC) &
+		    E1000_WUC_APME) != 0;
+	} else {
+		apme = error == E1000_SUCCESS &&
+		    (eeprom_data & apme_mask) != 0;
+	}
+	wol_capabilities = pci_has_pme(dev, PCI_POWERSTATE_D3_HOT) ?
+	    IFCAP_WOL : 0;
+	if (sc->hw.mac.type == e1000_82542 ||
+	    sc->hw.mac.type == e1000_82543)
+		wol_capabilities = 0;
+
+	/* APME selects the default; board and port restrictions select support. */
 	device_id = pci_get_device(dev);
 	switch (device_id) {
+	case E1000_DEV_ID_82542:
+	case E1000_DEV_ID_82543GC_FIBER:
+	case E1000_DEV_ID_82543GC_COPPER:
+	case E1000_DEV_ID_82541ER:
+	case E1000_DEV_ID_82541ER_LOM:
+	case E1000_DEV_ID_82544EI_FIBER:
+	case E1000_DEV_ID_82545EM_COPPER:
+	case E1000_DEV_ID_82545EM_FIBER:
+	case E1000_DEV_ID_82546EB_QUAD_COPPER:
+	case E1000_DEV_ID_82546GB_QUAD_COPPER:
 	case E1000_DEV_ID_82546GB_PCIE:
-		sc->wol = 0;
+		wol_capabilities = 0;
 		break;
 	case E1000_DEV_ID_82546EB_FIBER:
 	case E1000_DEV_ID_82546GB_FIBER:
-		/* Wake events only supported on port A for dual fiber
-		 * regardless of eeprom setting */
-		if (E1000_READ_REG(&sc->hw, E1000_STATUS) &
-		    E1000_STATUS_FUNC_1)
-			sc->wol = 0;
+		/*
+		 * Wake events are supported only on port A for dual fiber,
+		 * regardless of the NVM setting.
+		 */
+		if (sc->hw.bus.func == 1)
+			wol_capabilities = 0;
 		break;
 	case E1000_DEV_ID_82546GB_QUAD_COPPER_KSP3:
 		/* if quad port adapter, disable WoL on all but port A */
 		if (global_quad_port_a != 0)
-			sc->wol = 0;
+			wol_capabilities = 0;
+		else
+			wol_capabilities &= ~IFCAP_WOL_UCAST;
 		/* Reset for multiple quad port adapters */
 		if (++global_quad_port_a == 4)
 			global_quad_port_a = 0;
 		break;
+	case E1000_DEV_ID_82571EB_COPPER:
 	case E1000_DEV_ID_82571EB_FIBER:
-		/* Wake events only supported on port A for dual fiber
-		 * regardless of eeprom setting */
-		if (E1000_READ_REG(&sc->hw, E1000_STATUS) &
-		    E1000_STATUS_FUNC_1)
-			sc->wol = 0;
+	case E1000_DEV_ID_82571EB_SERDES:
+		/* These dual-port adapters support wake only on port A. */
+		if (sc->hw.bus.func == 1)
+			wol_capabilities = 0;
+		break;
+	case E1000_DEV_ID_82571EB_SERDES_QUAD:
+		wol_capabilities = 0;
 		break;
 	case E1000_DEV_ID_82571EB_QUAD_COPPER:
 	case E1000_DEV_ID_82571EB_QUAD_FIBER:
 	case E1000_DEV_ID_82571EB_QUAD_COPPER_LP:
+	case E1000_DEV_ID_82571PT_QUAD_COPPER:
 		/* if quad port adapter, disable WoL on all but port A */
 		if (global_quad_port_a != 0)
-			sc->wol = 0;
+			wol_capabilities = 0;
 		/* Reset for multiple quad port adapters */
 		if (++global_quad_port_a == 4)
 			global_quad_port_a = 0;
 		break;
+	case E1000_DEV_ID_82575GB_QUAD_COPPER:
+		wol_capabilities = 0;
+		break;
+	case E1000_DEV_ID_82575EB_FIBER_SERDES:
+	case E1000_DEV_ID_82576_FIBER:
+	case E1000_DEV_ID_82576_SERDES:
+		if (sc->hw.bus.func == 1)
+			wol_capabilities = 0;
+		break;
+	case E1000_DEV_ID_82576_QUAD_COPPER:
+	case E1000_DEV_ID_82576_QUAD_COPPER_ET2:
+		if (global_quad_port_a != 0)
+			wol_capabilities = 0;
+		if (++global_quad_port_a == 4)
+			global_quad_port_a = 0;
+		break;
+	default:
+		break;
 	}
-}
-
-
-/*
- * Enable PCI Wake On Lan capability
- */
-static void
+	/* Legacy and igb non-primary ports require an explicit NVM setting. */
+	if ((sc->hw.mac.type < e1000_82571 ||
+	    sc->hw.mac.type >= igb_mac_min) && sc->hw.bus.func != 0 &&
+	    !apme)
+		wol_capabilities = 0;
+
+	/* Some I350-family systems expose wake support but default it off. */
+	if ((sc->hw.mac.type == e1000_i350 &&
+	    pci_get_subvendor(dev) == EM_SUBVENDOR_HP) ||
+	    ((sc->hw.mac.type == e1000_i350 ||
+	    sc->hw.mac.type == e1000_i354) &&
+	    pci_get_subvendor(dev) == EM_SUBVENDOR_DELL) ||
+	    (sc->hw.mac.type == e1000_i350 &&
+	    ((pci_get_subdevice(dev) == EM_I350_SUBDEVICE_WOL_2 ||
+	    pci_get_subdevice(dev) == EM_I350_SUBDEVICE_WOL_3) &&
+	    sc->hw.bus.func == 0))) {
+		wol_capabilities = pci_has_pme(dev, PCI_POWERSTATE_D3_HOT) ?
+		    IFCAP_WOL : 0;
+		apme = false;
+	}
+	if (sc->hw.mac.type == e1000_i350 &&
+	    pci_get_subdevice(dev) == EM_I350_SUBDEVICE_WOL_1)
+		wol_capabilities = pci_has_pme(dev, PCI_POWERSTATE_D3_HOT) ?
+		    IFCAP_WOL : 0;
+
+	scctx->isc_capabilities &= ~IFCAP_WOL;
+	scctx->isc_capabilities |= wol_capabilities;
+	scctx->isc_capenable &= ~IFCAP_WOL;
+	if (wol_capabilities != 0 && apme)
+		scctx->isc_capenable |= IFCAP_WOL_MAGIC;
+}
+
+/* Configure the requested PCI Wake-on-LAN filters for suspend. */
+static int
 em_enable_wakeup(if_ctx_t ctx)
 {
 	struct e1000_softc *sc = iflib_get_softc(ctx);
 	device_t dev = iflib_get_dev(ctx);
 	if_t ifp = iflib_get_ifp(ctx);
-	int error = 0;
-	u32 ctrl, ctrl_ext, rctl;
+	int enabled, error = 0, master_error, mcnt;
+	u32 ctrl, ctrl_ext, rctl, saved_rctl, wuc, wufc;
+	bool manage, rctl_modified;
 
 	if (sc->vf_ifp)
-		return;
-	if (!pci_has_pm(dev))
-		return;
-
-	/*
-	 * Determine type of Wakeup: note that wol
-	 * is set with all bits on by default.
-	 */
-	if ((if_getcapenable(ifp) & IFCAP_WOL_MAGIC) == 0)
-		sc->wol &= ~E1000_WUFC_MAG;
-
-	if ((if_getcapenable(ifp) & IFCAP_WOL_UCAST) == 0)
-		sc->wol &= ~E1000_WUFC_EX;
+		return (0);
+	if (!pci_has_pme(dev, PCI_POWERSTATE_D3_HOT))
+		return (0);
 
-	if ((if_getcapenable(ifp) & IFCAP_WOL_MCAST) == 0)
-		sc->wol &= ~E1000_WUFC_MC;
-	else {
-		rctl = E1000_READ_REG(&sc->hw, E1000_RCTL);
-		rctl |= E1000_RCTL_MPE;
-		E1000_WRITE_REG(&sc->hw, E1000_RCTL, rctl);
+	enabled = if_getcapenable(ifp) & if_getcapabilities(ifp) & IFCAP_WOL;
+	manage = e1000_enable_mng_pass_thru(&sc->hw);
+	rctl_modified = false;
+	wuc = 0;
+	/* Early 82545EM/82546EB need APM clocks for D3 manageability. */
+	if (manage && (sc->hw.mac.type == e1000_82545 ||
+	    sc->hw.mac.type == e1000_82546))
+		wuc = E1000_WUC_APME;
+	wufc = 0;
+	if ((enabled & IFCAP_WOL_MAGIC) != 0)
+		wufc |= E1000_WUFC_MAG;
+	if ((enabled & IFCAP_WOL_UCAST) != 0)
+		wufc |= E1000_WUFC_EX;
+	if ((enabled & IFCAP_WOL_MCAST) != 0) {
+		wufc |= E1000_WUFC_MC;
+		bzero(sc->mta, ETHER_ADDR_LEN *
+		    MAX_NUM_MULTICAST_ADDRESSES);
+		mcnt = if_foreach_llmaddr(ifp, em_copy_maddr, sc->mta);
+		if (mcnt < MAX_NUM_MULTICAST_ADDRESSES) {
+			e1000_update_mc_addr_list(&sc->hw, sc->mta, mcnt);
+		} else {
+			switch (sc->hw.mac.type) {
+			case e1000_82544:
+			case e1000_82540:
+			case e1000_82545:
+			case e1000_82545_rev_3:
+			case e1000_82546:
+			case e1000_82546_rev_3:
+			case e1000_82541:
+			case e1000_82541_rev_2:
+			case e1000_82547:
+			case e1000_82547_rev_2:
+			case e1000_82575:
+			case e1000_82576:
+			case e1000_82580:
+				/* These parts require an MTA hit for WUFC_MC. */
+				em_fill_wakeup_mta(&sc->hw);
+				break;
+			default:
+				break;
+			}
+		}
 	}
 
-	if (!(sc->wol & (E1000_WUFC_EX | E1000_WUFC_MAG | E1000_WUFC_MC)))
+	if (wufc == 0) {
+		if (sc->hw.mac.type >= e1000_82544) {
+			E1000_WRITE_REG(&sc->hw, E1000_WUFC, 0);
+			E1000_WRITE_REG(&sc->hw, E1000_WUC, wuc);
+			E1000_WRITE_REG(&sc->hw, E1000_WUS, ~0U);
+		}
+		if (sc->wol_phy_wakeup && sc->wol_phy_armed)
+			(void)em_disable_phy_wakeup(sc, NULL);
+		if (manage) {
+			if (sc->suspend_link_powered_down)
+				em_power_up_wakeup_link(sc);
+			pci_enable_pme(dev);
+		} else {
+			em_power_down_wakeup_link(sc);
+			pci_clear_pme(dev);
+		}
+		goto master_disable;
+	}
+	bcopy(if_getlladdr(ifp), sc->hw.mac.addr, ETHER_ADDR_LEN);
+	error = e1000_rar_set(&sc->hw, sc->hw.mac.addr, 0);
+	if (error != E1000_SUCCESS) {
+		device_printf(dev,
+		    "Could not restore unicast wake address: %d\n", error);
 		goto pme;
+	}
+	saved_rctl = E1000_READ_REG(&sc->hw, E1000_RCTL);
+	rctl = saved_rctl;
+	rctl &= ~(E1000_RCTL_UPE | E1000_RCTL_MPE | E1000_RCTL_MO_3);
+	rctl |= E1000_RCTL_EN | E1000_RCTL_BAM |
+	    (sc->hw.mac.mc_filter_type << E1000_RCTL_MO_SHIFT);
+	if ((wufc & E1000_WUFC_MC) != 0)
+		rctl |= E1000_RCTL_MPE;
+	E1000_WRITE_REG(&sc->hw, E1000_RCTL, rctl);
+	rctl_modified = true;
 
 	/* Advertise the wakeup capability */
-	ctrl = E1000_READ_REG(&sc->hw, E1000_CTRL);
-	ctrl |= (E1000_CTRL_SWDPIN2 | E1000_CTRL_SWDPIN3);
-	E1000_WRITE_REG(&sc->hw, E1000_CTRL, ctrl);
-
-	/* Keep the laser running on Fiber adapters */
-	if (sc->hw.phy.media_type == e1000_media_type_fiber ||
-	    sc->hw.phy.media_type == e1000_media_type_internal_serdes) {
+	if (sc->hw.mac.type >= e1000_82540) {
+		ctrl = E1000_READ_REG(&sc->hw, E1000_CTRL);
+		ctrl |= E1000_CTRL_ADVD3WUC;
+		if (sc->hw.mac.type < igb_mac_min && !sc->wol_phy_wakeup)
+			ctrl |= E1000_CTRL_EN_PHY_PWR_MGMT;
+		E1000_WRITE_REG(&sc->hw, E1000_CTRL, ctrl);
+	}
+
+	/* Keep the laser running on legacy fiber and SerDes adapters. */
+	if (sc->hw.mac.type < igb_mac_min &&
+	    (sc->hw.phy.media_type == e1000_media_type_fiber ||
+	    sc->hw.phy.media_type == e1000_media_type_internal_serdes)) {
 		ctrl_ext = E1000_READ_REG(&sc->hw, E1000_CTRL_EXT);
 		ctrl_ext |= E1000_CTRL_EXT_SDP3_DATA;
 		E1000_WRITE_REG(&sc->hw, E1000_CTRL_EXT, ctrl_ext);
 	}
+	E1000_WRITE_REG(&sc->hw, E1000_WUS, ~0U);
+	em_power_up_wakeup_link(sc);
 
-	if ((sc->hw.mac.type == e1000_ich8lan) ||
-	    (sc->hw.mac.type == e1000_pchlan) ||
-	    (sc->hw.mac.type == e1000_ich9lan) ||
-	    (sc->hw.mac.type == e1000_ich10lan))
+	if (sc->hw.mac.type >= e1000_ich8lan &&
+	    sc->hw.mac.type < igb_mac_min)
 		e1000_suspend_workarounds_ich8lan(&sc->hw);
 
-	if ( sc->hw.mac.type >= e1000_pchlan) {
-		error = em_enable_phy_wakeup(sc);
+	if (sc->wol_phy_wakeup) {
+		error = em_enable_phy_wakeup(sc, wufc);
 		if (error)
 			goto pme;
 	} else {
 		/* Enable wakeup by the MAC */
-		E1000_WRITE_REG(&sc->hw, E1000_WUC, E1000_WUC_PME_EN);
-		E1000_WRITE_REG(&sc->hw, E1000_WUFC, sc->wol);
+		E1000_WRITE_REG(&sc->hw, E1000_WUC,
+		    wuc | E1000_WUC_PME_EN);
+		E1000_WRITE_REG(&sc->hw, E1000_WUFC, wufc);
 	}
 
-	if (sc->hw.phy.type == e1000_phy_igp_3)
+	/* The IGP3 D3 power-down workaround is specific to the em family. */
+	if (sc->hw.mac.type < igb_mac_min &&
+	    sc->hw.phy.type == e1000_phy_igp_3)
 		e1000_igp3_phy_powerdown_workaround_ich8lan(&sc->hw);
 
 pme:
-	if (!error && (if_getcapenable(ifp) & IFCAP_WOL))
+	if (!error)
 		pci_enable_pme(dev);
+	else {
+		E1000_WRITE_REG(&sc->hw, E1000_WUFC, 0);
+		E1000_WRITE_REG(&sc->hw, E1000_WUC, wuc);
+		if (rctl_modified)
+			E1000_WRITE_REG(&sc->hw, E1000_RCTL, saved_rctl);
+		pci_clear_pme(dev);
+	}
 
-	return;
+master_disable:
+	master_error = e1000_disable_pcie_master(&sc->hw);
+	if (master_error != E1000_SUCCESS)
+		device_printf(dev, "PCIe master disable timed out: %d\n",
+		    master_error);
+	master_error = pci_disable_busmaster(dev);
+	if (master_error != 0)
+		device_printf(dev, "PCI bus-master disable failed: %d\n",
+		    master_error);
+
+	return (error == E1000_SUCCESS ? 0 : EIO);
 }
 
-/*
- * WOL in the newer chipset interfaces (pchlan)
- * require thing to be copied into the phy
- */
+static void
+em_power_up_wakeup_link(struct e1000_softc *sc)
+{
+	struct e1000_hw *hw = &sc->hw;
+
+	if (hw->mac.type < igb_mac_min)
+		e1000_power_up_phy(hw);
+	else if (hw->phy.media_type == e1000_media_type_copper)
+		e1000_power_up_phy(hw);
+	else {
+		e1000_power_up_fiber_serdes_link(hw);
+		(void)e1000_setup_link(hw);
+	}
+	sc->suspend_link_powered_down = false;
+}
+
+/* Drop the unused suspend link through the controller's shared-code hook. */
+static void
+em_power_down_wakeup_link(struct e1000_softc *sc)
+{
+	struct e1000_hw *hw = &sc->hw;
+
+	if (hw->mac.type >= igb_mac_min &&
+	    hw->phy.media_type != e1000_media_type_copper)
+		e1000_shutdown_fiber_serdes_link(hw);
+	else
+		e1000_power_down_phy(hw);
+	sc->suspend_link_powered_down = true;
+}
+
+/* PCH PHY wake requires the MAC receive state on the BM wake page. */
 static int
-em_enable_phy_wakeup(struct e1000_softc *sc)
+em_enable_phy_wakeup(struct e1000_softc *sc, u32 wufc)
 {
 	struct e1000_hw *hw = &sc->hw;
-	u32 mreg, ret = 0;
-	u16 preg;
+	u32 mreg, wuc;
+	u16 preg, wuc_enable;
+	s32 error, restore_error;
+
+	/* Copy MAC RARs to PHY RARs before selecting the BM wake page. */
+	error = e1000_copy_rx_addrs_to_phy_ich8lan(hw);
+	if (error != E1000_SUCCESS)
+		goto out;
 
-	/* copy MAC RARs to PHY RARs */
-	e1000_copy_rx_addrs_to_phy_ich8lan(hw);
+	error = hw->phy.ops.acquire(hw);
+	if (error != E1000_SUCCESS) {
+		device_printf(sc->dev, "Could not acquire PHY for wakeup\n");
+		goto out;
+	}
+
+	error = e1000_enable_phy_wakeup_reg_access_bm(hw, &wuc_enable);
+	if (error != E1000_SUCCESS)
+		goto release;
+
+	/* Wake status is RW1C and survives controller reset. */
+	error = hw->phy.ops.write_reg_page(hw, BM_WUS, 0xffff);
+	if (error != E1000_SUCCESS)
+		goto restore;
 
 	/* copy MAC MTA to PHY MTA */
 	for (int i = 0; i < hw->mac.mta_reg_count; i++) {
 		mreg = E1000_READ_REG_ARRAY(hw, E1000_MTA, i);
-		e1000_write_phy_reg(hw, BM_MTA(i), (u16)(mreg & 0xFFFF));
-		e1000_write_phy_reg(hw, BM_MTA(i) + 1,
-		    (u16)((mreg >> 16) & 0xFFFF));
+		error = hw->phy.ops.write_reg_page(hw, BM_MTA(i),
+		    (u16)(mreg & 0xffff));
+		if (error != E1000_SUCCESS)
+			goto restore;
+		error = hw->phy.ops.write_reg_page(hw, BM_MTA(i) + 1,
+		    (u16)(mreg >> 16));
+		if (error != E1000_SUCCESS)
+			goto restore;
 	}
 
 	/* configure PHY Rx Control register */
-	e1000_read_phy_reg(hw, BM_RCTL, &preg);
+	error = hw->phy.ops.read_reg_page(hw, BM_RCTL, &preg);
+	if (error != E1000_SUCCESS)
+		goto restore;
 	mreg = E1000_READ_REG(hw, E1000_RCTL);
 	if (mreg & E1000_RCTL_UPE)
 		preg |= BM_RCTL_UPE;
@@ -6708,38 +7005,79 @@ em_enable_phy_wakeup(struct e1000_softc *sc)
 	mreg = E1000_READ_REG(hw, E1000_CTRL);
 	if (mreg & E1000_CTRL_RFCE)
 		preg |= BM_RCTL_RFCE;
-	e1000_write_phy_reg(hw, BM_RCTL, preg);
+	error = hw->phy.ops.write_reg_page(hw, BM_RCTL, preg);
+	if (error != E1000_SUCCESS)
+		goto restore;
+
+	wuc = E1000_WUC_PME_EN;
+	if ((wufc & (E1000_WUFC_MAG | E1000_WUFC_LNKC)) != 0)
+		wuc |= E1000_WUC_APME;
 
 	/* enable PHY wakeup in MAC register */
-	E1000_WRITE_REG(hw, E1000_WUC,
-	    E1000_WUC_PHY_WAKE | E1000_WUC_PME_EN | E1000_WUC_APME);
*** 128 LINES SKIPPED ***