From nobody Sun Aug 16 01:06:53 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hMyV56D51z6p8SH for ; Sun, 16 Aug 2026 01:06:53 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hMyV54NDtz3RgS for ; Sun, 16 Aug 2026 01:06:53 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786842413; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=HLTLNZW1phZ5pZme4uavtt3ilkcGOnGQxoIDIFE3nUA=; b=jA9cijGqYLZ0GDxwmOA98Gb1yBPFQJ0oy8CLCI0GWiziCdhABz+lLW96kU/BkOmKIRACYj ARydB3z3kiY1yClugKtKs748EHU+G5solxnNBG0Z6yN8boaxy17OBWL21DZsdRkDrYllA7 Q9AKZPpyVLzZfHhhg1gnIEFm/MMaeY/mYxr/4ZVN7jUWoEAKepfWfdyZxxJVAcQ/nOp7uo 904LqQeAM1YZmcwTjFr2txsWX3yfkqVu2XHM6INiWGIGYq51sY3SdouQxj+aFGRSv6EFN0 10SGdWpBBEq/cLbXRybwKuJLQmqCiGIkQAcJPo4Xxr+9RUKWpmL3MYYC6sxnSQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1786842413; a=rsa-sha256; cv=none; b=pL253sqlzc5+X3HpLC6AQrUqQ45ZF8eiaxoHG0N21fqM9db1bsso1pEZYK2eXAroxxR0V0 TkF3DQtu6j69oZRjfyIqmjeya6j+z2XUzmaXSAX3hRtTqyBZH7d1bx5casd0hrTrkvW3Xu obPIREkRReYBOGO3bwMH5lMicyKgmGesYTb6JEeIvkTo2zZWH9svNXmQ+RLpD+8+TUcBdx jR8GY3QctnDkQzYkRhtQ6xV68u0A8jl2E1ypuv4OMN455xtq1TfURwWwodPVEGNNQhkb9f xlqr2iVgIIwAtAzFQJD81fpT15ujqPakRSqX/2aMXWi/pK18eMay70PlT4sBsw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786842413; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=HLTLNZW1phZ5pZme4uavtt3ilkcGOnGQxoIDIFE3nUA=; b=dq9u0nf+gbHNYHu72ijhLigZZL91fsxvqctn0gp/Qzw+41MvG4gTpTo/sgv18jY4cUkWB9 UbVpDVEpDbGyBVYaP7If56/plvAqnATNBVDfpqRbp20wtzU7CUD4IW13zkJokb7zUIw7LU IBJpTmZn/otT69kaFWctEl6NT6vFs8uL5CcoqC7/8sQZRbJewEbR9dOmtJ//ALuzDb0KPJ 3wFyr2/a0qkLxxV2g/x18dxsdsyc6caeXYBZp5ntv3+5NinOzb8PZPy/dAhvsufFotEdbz nVBNkcmdjiYZ4EeO3IkPhRG3f1OdHMjVh44ZML2yysLn3TPCWa+W+EKkAvn9kQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hMyV52ZyPz14q2 for ; Sun, 16 Aug 2026 01:06:53 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 36efd by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Sun, 16 Aug 2026 01:06:53 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Kevin Bowling Subject: git: 40367d1f3b8f - main - ixv: Support E610 mailbox API 1.6 List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kbowling X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 40367d1f3b8ff694ffc3e4bf3278ec2a762648d2 Auto-Submitted: auto-generated Date: Sun, 16 Aug 2026 01:06:53 +0000 Message-Id: <6a810d2d.36efd.4415ac40@gitrepo.freebsd.org> The branch main has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=40367d1f3b8ff694ffc3e4bf3278ec2a762648d2 commit 40367d1f3b8ff694ffc3e4bf3278ec2a762648d2 Author: Kevin Bowling AuthorDate: 2026-08-01 02:41:56 +0000 Commit: Kevin Bowling CommitDate: 2026-08-16 01:06:42 +0000 ixv: Support E610 mailbox API 1.6 E610 VFs no longer report the actual PF link state and speed through VFLINKS. They can consequently report the default 10 Gb/s speed even when the physical link uses another rate. Negotiate mailbox API 1.6 on E610 and request the PF link state with its three-dword operation. Retain VFLINKS as the fallback when an older PF rejects API 1.6. Permit API 1.6 in the inherited xcast and queue discovery helpers so negotiating the newer revision does not disable existing operations. Use GET_QUEUES to replace E610's one-queue fallback with the grant from the PF. The common path continues to use one iflib queue set per data MSI-X vector and caps the result at two queue pairs. Preserve mailbox transport errors so the driver can distinguish an explicit PF NACK from a transient timeout. A NACK means clear-to-send state was lost and requires a VF reset. Preserve the last confirmed link state across brief transport failures and publish link down after three consecutive failures. Poll E610 link state every two seconds, matching Intel's ixgbevf service timer, and phase VFs across the intervening iflib timer ticks. This avoids a mailbox polling herd when many VFs share a PF. Media-status queries return the cached state instead of starting another synchronous exchange. An admin interrupt caused by a mailbox reply only checks for an unsolicited PF reset, preventing a request/reply interrupt loop. Hardware validation on an E610 10GBASE-T PF exercised 63 VFs. Each VF negotiated API 1.6, two queue pairs, and three MSI-X vectors. Phased polling kept 31 active VFs idle, and all 63 recovered after a PF down/up cycle without watchdogs. Adapt the API 1.6 link-state operation from DPDK shared ixgbe code. The timeout and NACK distinction follows Intel ixgbevf 5.3.25. MFC after: 2 weeks Sponsored by: Dirk-Willem van Gulik from Web Weaving (E610 hardware) Sponsored by: BBOX.io --- sys/dev/ixgbe/if_ixv.c | 64 +++++++++++++++++++++++++++++++++++++++++++---- sys/dev/ixgbe/ixgbe.h | 3 +++ sys/dev/ixgbe/ixgbe_mbx.h | 3 +++ sys/dev/ixgbe/ixgbe_vf.c | 60 ++++++++++++++++++++++++++++++++++++++++++++ sys/dev/ixgbe/ixgbe_vf.h | 2 ++ 5 files changed, 127 insertions(+), 5 deletions(-) diff --git a/sys/dev/ixgbe/if_ixv.c b/sys/dev/ixgbe/if_ixv.c index 93af5daa01d3..99d1169254c5 100644 --- a/sys/dev/ixgbe/if_ixv.c +++ b/sys/dev/ixgbe/if_ixv.c @@ -56,6 +56,11 @@ static const sbintime_t ixv_mbx_retry_delay[] = { static const struct timeval ixv_mbx_log_interval = { 60, 0 }; +/* Bound stale carrier state without flapping on a busy PF mailbox. */ +#define IXV_LINK_MBX_FAILURE_LIMIT 3 +/* Match Intel's two-second VF service timer and spread PF mailbox load. */ +#define IXV_LINK_POLL_TICKS 4 + /************************************************************************ * PCI Device ID Table * @@ -448,6 +453,7 @@ ixv_if_attach_pre(if_ctx_t ctx) /* Determine hardware revision */ ixv_identify_hardware(ctx); ixv_init_device_features(sc); + sc->vf_link_poll_tick = device_get_unit(dev) % IXV_LINK_POLL_TICKS; /* Initialize the shared code */ error = ixgbe_init_ops_vf(hw); @@ -896,6 +902,9 @@ ixv_mbx_retry_succeeded(struct ixgbe_softc *sc) if (sc->vf_mbx_retry_initialized) callout_stop(&sc->vf_mbx_retry); sc->vf_mbx_retry_stage = 0; + sc->vf_link_mbx_failures = 0; + sc->vf_link_poll_tick = + device_get_unit(sc->dev) % IXV_LINK_POLL_TICKS; sc->vf_mbx_last_log.tv_sec = 0; sc->vf_mbx_last_log.tv_usec = 0; } @@ -982,7 +991,12 @@ ixv_if_media_status(if_ctx_t ctx, struct ifmediareq * ifmr) INIT_DEBUGOUT("ixv_media_status: begin"); - iflib_admin_intr_deferred(ctx); + /* E610 link state is refreshed by the phased service timer. */ + if (sc->hw.mac.type != ixgbe_mac_E610_vf || + sc->hw.api_version != ixgbe_mbox_api_16) { + atomic_set_32(&sc->vf_link_update, 1); + iflib_admin_intr_deferred(ctx); + } ifmr->ifm_status = IFM_AVALID; ifmr->ifm_active = IFM_ETHER; @@ -1062,6 +1076,10 @@ ixv_negotiate_api(struct ixgbe_softc *sc) }; int i = 0; + if (hw->mac.type == ixgbe_mac_E610_vf && + ixgbevf_negotiate_api_version(hw, ixgbe_mbox_api_16) == 0) + return (0); + while (mbx_api[i] != ixgbe_mbox_api_unknown) { if (ixgbevf_negotiate_api_version(hw, mbx_api[i]) == 0) return (0); @@ -1095,6 +1113,9 @@ ixv_queue_limit(struct ixgbe_softc *sc, bool mailbox_ready) case ixgbe_mac_X550EM_a_vf: limit = 2; break; + case ixgbe_mac_E610_vf: + limit = 1; + break; default: return (1); } @@ -1105,6 +1126,7 @@ ixv_queue_limit(struct ixgbe_softc *sc, bool mailbox_ready) case ixgbe_mbox_api_11: case ixgbe_mbox_api_12: case ixgbe_mbox_api_13: + case ixgbe_mbox_api_16: num_tcs = default_tc = 0; if (ixgbevf_get_queues(hw, &num_tcs, &default_tc) == 0) { limit = imin(hw->mac.max_tx_queues, @@ -1248,6 +1270,12 @@ ixv_if_local_timer(if_ctx_t ctx, uint16_t qid) sc = iflib_get_softc(ctx); atomic_set_32(&sc->vf_vlan_retry_tick, 1); + if (sc->hw.mac.type != ixgbe_mac_E610_vf || + sc->hw.api_version != ixgbe_mbox_api_16 || + ++sc->vf_link_poll_tick == IXV_LINK_POLL_TICKS) { + sc->vf_link_poll_tick = 0; + atomic_set_32(&sc->vf_link_update, 1); + } /* Fire off the adminq task */ iflib_admin_intr_deferred(ctx); @@ -1266,6 +1294,7 @@ ixv_if_update_admin_status(if_ctx_t ctx) struct ixgbe_softc *sc = iflib_get_softc(ctx); device_t dev = iflib_get_dev(ctx); if_t ifp = iflib_get_ifp(ctx); + bool check_link; s32 status; uint64_t baudrate; @@ -1278,10 +1307,34 @@ ixv_if_update_admin_status(if_ctx_t ctx) return; } - sc->hw.mac.get_link_status = true; - - status = ixgbe_check_link(&sc->hw, &sc->link_speed, - &sc->link_up, false); + check_link = atomic_readandclear_32(&sc->vf_link_update) != 0; + if (sc->hw.mac.type != ixgbe_mac_E610_vf || + sc->hw.api_version != ixgbe_mbox_api_16) + check_link = true; + if (check_link) { + sc->hw.mac.get_link_status = true; + status = ixgbe_check_link(&sc->hw, &sc->link_speed, + &sc->link_up, false); + } else if (ixgbe_check_for_rst(&sc->hw, 0) == IXGBE_SUCCESS) { + /* Process an unsolicited PF reset without issuing another query. */ + status = IXGBE_ERR_MBX; + } else + status = IXGBE_SUCCESS; + if (sc->hw.mac.type == ixgbe_mac_E610_vf && + sc->hw.api_version == ixgbe_mbox_api_16 && + status != IXGBE_SUCCESS && status != IXGBE_ERR_MBX) { + /* + * A busy PF can miss an individual link-state request. Preserve + * the last confirmed state across brief transport failures, but + * bound how long stale carrier can remain visible if the PF is gone. + */ + if (sc->vf_link_mbx_failures < IXV_LINK_MBX_FAILURE_LIMIT) + sc->vf_link_mbx_failures++; + if (sc->vf_link_mbx_failures == IXV_LINK_MBX_FAILURE_LIMIT) + sc->link_up = false; + status = IXGBE_SUCCESS; + } else if (status == IXGBE_SUCCESS) + sc->vf_link_mbx_failures = 0; if (status != IXGBE_SUCCESS && sc->hw.adapter_stopped == false) { /* Mailbox's Clear To Send status is lost or timeout occurred. @@ -1345,6 +1398,7 @@ ixv_if_stop(if_ctx_t ctx) if (mailbox_ready && (if_getflags(ifp) & IFF_UP) == 0) hw->mac.ops.reset_hw(hw); atomic_store_rel_32(&sc->vf_mbx_ready, 0); + sc->vf_link_mbx_failures = 0; sc->hw.adapter_stopped = false; hw->mac.ops.stop_adapter(hw); diff --git a/sys/dev/ixgbe/ixgbe.h b/sys/dev/ixgbe/ixgbe.h index 8ded6fa41c65..f4a49516c76e 100644 --- a/sys/dev/ixgbe/ixgbe.h +++ b/sys/dev/ixgbe/ixgbe.h @@ -428,10 +428,13 @@ struct ixgbe_softc { u32 vf_mbx_ready; u32 vf_mbx_retry_pending; u32 vf_vlan_retry_tick; + u32 vf_link_update; u16 vf_vlan_retry_cursor; u8 vf_mbx_retry_stage; bool vf_mbx_retry_initialized; bool vf_mcast_overflow_warned; + u8 vf_link_mbx_failures; + u8 vf_link_poll_tick; /* Info about the interface */ int advertise; /* link speeds */ diff --git a/sys/dev/ixgbe/ixgbe_mbx.h b/sys/dev/ixgbe/ixgbe_mbx.h index 827e74a2e397..b719d0c83c1a 100644 --- a/sys/dev/ixgbe/ixgbe_mbx.h +++ b/sys/dev/ixgbe/ixgbe_mbx.h @@ -153,6 +153,9 @@ enum ixgbe_pfvf_api_rev { #define IXGBE_VF_UPDATE_XCAST_MODE 0x0c #define IXGBE_VF_GET_LINK_STATE 0x10 +/* mailbox API, version 1.6 VF requests */ +#define IXGBE_VF_GET_PF_LINK_STATE 0x11 /* get status of the link on PF */ + /* mode choices for IXGBE_VF_UPDATE_XCAST_MODE */ enum ixgbevf_xcast_modes { IXGBEVF_XCAST_MODE_NONE = 0, diff --git a/sys/dev/ixgbe/ixgbe_vf.c b/sys/dev/ixgbe/ixgbe_vf.c index 00f0cb131cbf..78588c63030b 100644 --- a/sys/dev/ixgbe/ixgbe_vf.c +++ b/sys/dev/ixgbe/ixgbe_vf.c @@ -444,6 +444,7 @@ s32 ixgbevf_update_xcast_mode(struct ixgbe_hw *hw, int xcast_mode) /* Fall through */ case ixgbe_mbox_api_13: case ixgbe_mbox_api_15: + case ixgbe_mbox_api_16: break; default: return IXGBE_ERR_FEATURE_NOT_SUPPORTED; @@ -490,6 +491,47 @@ s32 ixgbe_get_link_state_vf(struct ixgbe_hw *hw, bool *link_state) return ret_val; } +/** + * ixgbevf_get_pf_link_state - Get PF's link status + * @hw: pointer to the HW structure + * @speed: link speed + * @link_up: indicate if link is up/down + * + * Ask PF to provide link_up state and speed of the link. + * + * Return: IXGBE_ERR_MBX in the case of mailbox error, + * IXGBE_ERR_FEATURE_NOT_SUPPORTED if the op is not supported or 0 on success. + */ +int ixgbevf_get_pf_link_state(struct ixgbe_hw *hw, ixgbe_link_speed *speed, + bool *link_up) +{ + u32 msgbuf[3] = {}; + int err; + + if (hw->api_version != ixgbe_mbox_api_16) + return IXGBE_ERR_FEATURE_NOT_SUPPORTED; + + msgbuf[0] = IXGBE_VF_GET_PF_LINK_STATE; + err = ixgbevf_write_msg_read_ack(hw, msgbuf, msgbuf, 3); + if (err) + return (err); + if (msgbuf[0] & IXGBE_VT_MSGTYPE_FAILURE) { + *speed = IXGBE_LINK_SPEED_UNKNOWN; + return (IXGBE_ERR_MBX); + } + if ((msgbuf[0] & IXGBE_VT_MSGTYPE_CTS) == 0 || + (msgbuf[0] & ~IXGBE_VT_MSGTYPE_CTS) != + (IXGBE_VF_GET_PF_LINK_STATE | IXGBE_VT_MSGTYPE_SUCCESS)) { + *speed = IXGBE_LINK_SPEED_UNKNOWN; + return (IXGBE_ERR_MBX); + } + + *speed = msgbuf[1]; + *link_up = msgbuf[2]; + + return err; +} + /** * ixgbe_set_vfta_vf - Set/Unset vlan filter table address * @hw: pointer to the HW structure @@ -623,12 +665,29 @@ s32 ixgbe_check_mac_link_vf(struct ixgbe_hw *hw, ixgbe_link_speed *speed, UNREFERENCED_1PARAMETER(autoneg_wait_to_complete); + if (link_up == NULL) + return (IXGBE_ERR_PARAM); + /* If we were hit with a reset drop the link */ if (!mbx->ops[0].check_for_rst(hw, 0) || !mbx->timeout) mac->get_link_status = true; if (!mac->get_link_status) goto out; + if (mac->type == ixgbe_mac_E610_vf && + hw->api_version == ixgbe_mbox_api_16) { + ret_val = ixgbevf_get_pf_link_state(hw, speed, link_up); + if (ret_val == IXGBE_ERR_MBX) { + /* A PF NACK means the VF has lost clear-to-send state. */ + *link_up = false; + return (ret_val); + } else if (ret_val != IXGBE_SUCCESS) { + /* Let the driver debounce transient transport failures. */ + return (ret_val); + } + mac->get_link_status = !*link_up; + return (IXGBE_SUCCESS); + } /* if link status is down no point in checking to see if pf is up */ links_reg = IXGBE_READ_REG(hw, IXGBE_VFLINKS); @@ -777,6 +836,7 @@ int ixgbevf_get_queues(struct ixgbe_hw *hw, unsigned int *num_tcs, case ixgbe_mbox_api_12: case ixgbe_mbox_api_13: case ixgbe_mbox_api_15: + case ixgbe_mbox_api_16: break; default: return 0; diff --git a/sys/dev/ixgbe/ixgbe_vf.h b/sys/dev/ixgbe/ixgbe_vf.h index 2e8921a21fed..7f5a8c2e85cf 100644 --- a/sys/dev/ixgbe/ixgbe_vf.h +++ b/sys/dev/ixgbe/ixgbe_vf.h @@ -135,6 +135,8 @@ s32 ixgbe_update_mc_addr_list_vf(struct ixgbe_hw *hw, u8 *mc_addr_list, bool clear); s32 ixgbevf_update_xcast_mode(struct ixgbe_hw *hw, int xcast_mode); s32 ixgbe_get_link_state_vf(struct ixgbe_hw *hw, bool *link_state); +int ixgbevf_get_pf_link_state(struct ixgbe_hw *hw, ixgbe_link_speed *speed, + bool *link_up); s32 ixgbe_set_vfta_vf(struct ixgbe_hw *hw, u32 vlan, u32 vind, bool vlan_on, bool vlvf_bypass); s32 ixgbevf_rlpml_set_vf(struct ixgbe_hw *hw, u16 max_size);