From nobody Sun Aug 30 00:30:35 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hXY1m0Gc0z6pjtr for ; Sun, 30 Aug 2026 00:30:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hXY1l6L8kz3YjF for ; Sun, 30 Aug 2026 00:30:35 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1788049835; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=25spJ7krMc3cHr2B3hstpRBgAYKH/cvsoIdSkmDTfqM=; b=gnGFlh+pP3TN/L7T6n1nqdd5DR+e5EJngZU48mLQydqutGQzI6rBgWf+iYmikCTRAzGIvX whdgD3ehj1KmagMdC1PtYrAm3//f0m8YpixpuRJ8cC7rNeudgBPPEmsXyGjTnQvM3RdFl2 d8y4K3KQSe7aqclLZ2AcBml98hUCKQLB9bPRizPHJljc3DPzBbMqTIHsWTKDvbmrVeb72A Njw4aHfCdndYWsAv+G+y3AzkyzVDdJXPanfIG1gXT4yhMincYbN0Q0yDFkh/buxp5bSeru wGqbxH2Rc28vagHY3Yj4VXJ45OJ4u13Vw8TlXaJwlq5HlfTeRSaHnJx0Np/c2g== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1788049835; a=rsa-sha256; cv=none; b=bmonujEsS4hEIN/B0te2V4jVSsBKe9qtqjDVH5Z3Y2ErsrPLBtsgK7p0UA+SQjkUjMXPpt p54wrRDFWv4gwrp5G0Hbfhra1MkpxcEN/vb4NS5Nq+v325Jx+Uko1kUxUT+4Q1bQetV8Vx Unv1S6tVUO7lW+BRK2i1uwjrP2OtZbERWP4f3ZQg1c+DMj62dFzn9pAgQHJ8pE6yB4v4RQ La3pSfqCS1L+VIk/9/EXLKhKMi6E1MlA0qVfEDr2p0Vt4OkDAhFJ+g2SWL/IPY5aOq7d40 jP+cyO5ZNNDDZU0ers5wwLuSOUmUrG9XQw6M7oFisV7OEWgTozeXLZ0c7BraQg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1788049835; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=25spJ7krMc3cHr2B3hstpRBgAYKH/cvsoIdSkmDTfqM=; b=WrBkIWM1pyGOu6p1UjxDEuB4t683l5uu+mNdQG9lwJUKOJvsxJWrKA1HzOXuPNyFopJe5P 2sQRm97a9Os8HBgBccUYpGYtiEAzaeOAxeiUb5mH3/5RIAy/huRLBEm+Lr2sw4hFwN5hsP jdHY8y+NBDEba49SjsAKXvU3mTnKp4PB2oJKKTDLqpGRvU9WpTBwYvuZMsnFO11v2gAAN6 CTpWSvyqYxuAf9M9rEW9KFkj7+Zct1qX4qmH/uI5OSIul+CfN2DQCE/lIWg+YT98Xkc88q B59i1qBcQZTDOI1M08aPp5ayg6plGi7UvYxIzZaSlwTyOXWQhFViqSFE1fVfNg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hXY1l1jbQz12bv for ; Sun, 30 Aug 2026 00:30:35 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 395a0 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Sun, 30 Aug 2026 00:30:35 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kevin Bowling Subject: git: b36753db40ca - stable/14 - ixv: Support E610 mailbox API 1.6 List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kbowling X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: b36753db40ca462a1847e5ba7bc76fd3ee3e240e Auto-Submitted: auto-generated Date: Sun, 30 Aug 2026 00:30:35 +0000 Message-Id: <6a9379ab.395a0.4e0c71b0@gitrepo.freebsd.org> The branch stable/14 has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=b36753db40ca462a1847e5ba7bc76fd3ee3e240e commit b36753db40ca462a1847e5ba7bc76fd3ee3e240e Author: Kevin Bowling AuthorDate: 2026-08-01 02:41:56 +0000 Commit: Kevin Bowling CommitDate: 2026-08-30 00:29:39 +0000 ixv: Support E610 mailbox API 1.6 E610 VFs no longer report the actual PF link state and speed through VFLINKS. They can consequently report the default 10 Gb/s speed even when the physical link uses another rate. Negotiate mailbox API 1.6 on E610 and request the PF link state with its three-dword operation. Retain VFLINKS as the fallback when an older PF rejects API 1.6. Permit API 1.6 in the inherited xcast and queue discovery helpers so negotiating the newer revision does not disable existing operations. Use GET_QUEUES to replace E610's one-queue fallback with the grant from the PF. The common path continues to use one iflib queue set per data MSI-X vector and caps the result at two queue pairs. Preserve mailbox transport errors so the driver can distinguish an explicit PF NACK from a transient timeout. A NACK means clear-to-send state was lost and requires a VF reset. Preserve the last confirmed link state across brief transport failures and publish link down after three consecutive failures. Poll E610 link state every two seconds, matching Intel's ixgbevf service timer, and phase VFs across the intervening iflib timer ticks. This avoids a mailbox polling herd when many VFs share a PF. Media-status queries return the cached state instead of starting another synchronous exchange. An admin interrupt caused by a mailbox reply only checks for an unsolicited PF reset, preventing a request/reply interrupt loop. Hardware validation on an E610 10GBASE-T PF exercised 63 VFs. Each VF negotiated API 1.6, two queue pairs, and three MSI-X vectors. Phased polling kept 31 active VFs idle, and all 63 recovered after a PF down/up cycle without watchdogs. Adapt the API 1.6 link-state operation from DPDK shared ixgbe code. The timeout and NACK distinction follows Intel ixgbevf 5.3.25. Sponsored by: Dirk-Willem van Gulik from Web Weaving (E610 hardware) Sponsored by: BBOX.io (cherry picked from commit 40367d1f3b8ff694ffc3e4bf3278ec2a762648d2) --- sys/dev/ixgbe/if_ixv.c | 64 +++++++++++++++++++++++++++++++++++++++++++---- sys/dev/ixgbe/ixgbe.h | 3 +++ sys/dev/ixgbe/ixgbe_mbx.h | 3 +++ sys/dev/ixgbe/ixgbe_vf.c | 60 ++++++++++++++++++++++++++++++++++++++++++++ sys/dev/ixgbe/ixgbe_vf.h | 2 ++ 5 files changed, 127 insertions(+), 5 deletions(-) diff --git a/sys/dev/ixgbe/if_ixv.c b/sys/dev/ixgbe/if_ixv.c index a76fa913037b..6e9aa0d14ec7 100644 --- a/sys/dev/ixgbe/if_ixv.c +++ b/sys/dev/ixgbe/if_ixv.c @@ -56,6 +56,11 @@ static const sbintime_t ixv_mbx_retry_delay[] = { static const struct timeval ixv_mbx_log_interval = { 60, 0 }; +/* Bound stale carrier state without flapping on a busy PF mailbox. */ +#define IXV_LINK_MBX_FAILURE_LIMIT 3 +/* Match Intel's two-second VF service timer and spread PF mailbox load. */ +#define IXV_LINK_POLL_TICKS 4 + /************************************************************************ * PCI Device ID Table * @@ -448,6 +453,7 @@ ixv_if_attach_pre(if_ctx_t ctx) /* Determine hardware revision */ ixv_identify_hardware(ctx); ixv_init_device_features(sc); + sc->vf_link_poll_tick = device_get_unit(dev) % IXV_LINK_POLL_TICKS; /* Initialize the shared code */ error = ixgbe_init_ops_vf(hw); @@ -896,6 +902,9 @@ ixv_mbx_retry_succeeded(struct ixgbe_softc *sc) if (sc->vf_mbx_retry_initialized) callout_stop(&sc->vf_mbx_retry); sc->vf_mbx_retry_stage = 0; + sc->vf_link_mbx_failures = 0; + sc->vf_link_poll_tick = + device_get_unit(sc->dev) % IXV_LINK_POLL_TICKS; sc->vf_mbx_last_log.tv_sec = 0; sc->vf_mbx_last_log.tv_usec = 0; } @@ -982,7 +991,12 @@ ixv_if_media_status(if_ctx_t ctx, struct ifmediareq * ifmr) INIT_DEBUGOUT("ixv_media_status: begin"); - iflib_admin_intr_deferred(ctx); + /* E610 link state is refreshed by the phased service timer. */ + if (sc->hw.mac.type != ixgbe_mac_E610_vf || + sc->hw.api_version != ixgbe_mbox_api_16) { + atomic_set_32(&sc->vf_link_update, 1); + iflib_admin_intr_deferred(ctx); + } ifmr->ifm_status = IFM_AVALID; ifmr->ifm_active = IFM_ETHER; @@ -1062,6 +1076,10 @@ ixv_negotiate_api(struct ixgbe_softc *sc) }; int i = 0; + if (hw->mac.type == ixgbe_mac_E610_vf && + ixgbevf_negotiate_api_version(hw, ixgbe_mbox_api_16) == 0) + return (0); + while (mbx_api[i] != ixgbe_mbox_api_unknown) { if (ixgbevf_negotiate_api_version(hw, mbx_api[i]) == 0) return (0); @@ -1095,6 +1113,9 @@ ixv_queue_limit(struct ixgbe_softc *sc, bool mailbox_ready) case ixgbe_mac_X550EM_a_vf: limit = 2; break; + case ixgbe_mac_E610_vf: + limit = 1; + break; default: return (1); } @@ -1105,6 +1126,7 @@ ixv_queue_limit(struct ixgbe_softc *sc, bool mailbox_ready) case ixgbe_mbox_api_11: case ixgbe_mbox_api_12: case ixgbe_mbox_api_13: + case ixgbe_mbox_api_16: num_tcs = default_tc = 0; if (ixgbevf_get_queues(hw, &num_tcs, &default_tc) == 0) { limit = imin(hw->mac.max_tx_queues, @@ -1248,6 +1270,12 @@ ixv_if_local_timer(if_ctx_t ctx, uint16_t qid) sc = iflib_get_softc(ctx); atomic_set_32(&sc->vf_vlan_retry_tick, 1); + if (sc->hw.mac.type != ixgbe_mac_E610_vf || + sc->hw.api_version != ixgbe_mbox_api_16 || + ++sc->vf_link_poll_tick == IXV_LINK_POLL_TICKS) { + sc->vf_link_poll_tick = 0; + atomic_set_32(&sc->vf_link_update, 1); + } /* Fire off the adminq task */ iflib_admin_intr_deferred(ctx); @@ -1266,6 +1294,7 @@ ixv_if_update_admin_status(if_ctx_t ctx) struct ixgbe_softc *sc = iflib_get_softc(ctx); device_t dev = iflib_get_dev(ctx); if_t ifp = iflib_get_ifp(ctx); + bool check_link; s32 status; uint64_t baudrate; @@ -1278,10 +1307,34 @@ ixv_if_update_admin_status(if_ctx_t ctx) return; } - sc->hw.mac.get_link_status = true; - - status = ixgbe_check_link(&sc->hw, &sc->link_speed, - &sc->link_up, false); + check_link = atomic_readandclear_32(&sc->vf_link_update) != 0; + if (sc->hw.mac.type != ixgbe_mac_E610_vf || + sc->hw.api_version != ixgbe_mbox_api_16) + check_link = true; + if (check_link) { + sc->hw.mac.get_link_status = true; + status = ixgbe_check_link(&sc->hw, &sc->link_speed, + &sc->link_up, false); + } else if (ixgbe_check_for_rst(&sc->hw, 0) == IXGBE_SUCCESS) { + /* Process an unsolicited PF reset without issuing another query. */ + status = IXGBE_ERR_MBX; + } else + status = IXGBE_SUCCESS; + if (sc->hw.mac.type == ixgbe_mac_E610_vf && + sc->hw.api_version == ixgbe_mbox_api_16 && + status != IXGBE_SUCCESS && status != IXGBE_ERR_MBX) { + /* + * A busy PF can miss an individual link-state request. Preserve + * the last confirmed state across brief transport failures, but + * bound how long stale carrier can remain visible if the PF is gone. + */ + if (sc->vf_link_mbx_failures < IXV_LINK_MBX_FAILURE_LIMIT) + sc->vf_link_mbx_failures++; + if (sc->vf_link_mbx_failures == IXV_LINK_MBX_FAILURE_LIMIT) + sc->link_up = false; + status = IXGBE_SUCCESS; + } else if (status == IXGBE_SUCCESS) + sc->vf_link_mbx_failures = 0; if (status != IXGBE_SUCCESS && sc->hw.adapter_stopped == false) { /* Mailbox's Clear To Send status is lost or timeout occurred. @@ -1345,6 +1398,7 @@ ixv_if_stop(if_ctx_t ctx) if (mailbox_ready && (if_getflags(ifp) & IFF_UP) == 0) hw->mac.ops.reset_hw(hw); atomic_store_rel_32(&sc->vf_mbx_ready, 0); + sc->vf_link_mbx_failures = 0; sc->hw.adapter_stopped = false; hw->mac.ops.stop_adapter(hw); diff --git a/sys/dev/ixgbe/ixgbe.h b/sys/dev/ixgbe/ixgbe.h index 43610e863c6d..405d5b5daf14 100644 --- a/sys/dev/ixgbe/ixgbe.h +++ b/sys/dev/ixgbe/ixgbe.h @@ -428,10 +428,13 @@ struct ixgbe_softc { u32 vf_mbx_ready; u32 vf_mbx_retry_pending; u32 vf_vlan_retry_tick; + u32 vf_link_update; u16 vf_vlan_retry_cursor; u8 vf_mbx_retry_stage; bool vf_mbx_retry_initialized; bool vf_mcast_overflow_warned; + u8 vf_link_mbx_failures; + u8 vf_link_poll_tick; /* Info about the interface */ int advertise; /* link speeds */ diff --git a/sys/dev/ixgbe/ixgbe_mbx.h b/sys/dev/ixgbe/ixgbe_mbx.h index fc6831f67b34..1d59a0dab1cd 100644 --- a/sys/dev/ixgbe/ixgbe_mbx.h +++ b/sys/dev/ixgbe/ixgbe_mbx.h @@ -152,6 +152,9 @@ enum ixgbe_pfvf_api_rev { #define IXGBE_VF_UPDATE_XCAST_MODE 0x0c #define IXGBE_VF_GET_LINK_STATE 0x10 +/* mailbox API, version 1.6 VF requests */ +#define IXGBE_VF_GET_PF_LINK_STATE 0x11 /* get status of the link on PF */ + /* mode choices for IXGBE_VF_UPDATE_XCAST_MODE */ enum ixgbevf_xcast_modes { IXGBEVF_XCAST_MODE_NONE = 0, diff --git a/sys/dev/ixgbe/ixgbe_vf.c b/sys/dev/ixgbe/ixgbe_vf.c index 00f0cb131cbf..78588c63030b 100644 --- a/sys/dev/ixgbe/ixgbe_vf.c +++ b/sys/dev/ixgbe/ixgbe_vf.c @@ -444,6 +444,7 @@ s32 ixgbevf_update_xcast_mode(struct ixgbe_hw *hw, int xcast_mode) /* Fall through */ case ixgbe_mbox_api_13: case ixgbe_mbox_api_15: + case ixgbe_mbox_api_16: break; default: return IXGBE_ERR_FEATURE_NOT_SUPPORTED; @@ -490,6 +491,47 @@ s32 ixgbe_get_link_state_vf(struct ixgbe_hw *hw, bool *link_state) return ret_val; } +/** + * ixgbevf_get_pf_link_state - Get PF's link status + * @hw: pointer to the HW structure + * @speed: link speed + * @link_up: indicate if link is up/down + * + * Ask PF to provide link_up state and speed of the link. + * + * Return: IXGBE_ERR_MBX in the case of mailbox error, + * IXGBE_ERR_FEATURE_NOT_SUPPORTED if the op is not supported or 0 on success. + */ +int ixgbevf_get_pf_link_state(struct ixgbe_hw *hw, ixgbe_link_speed *speed, + bool *link_up) +{ + u32 msgbuf[3] = {}; + int err; + + if (hw->api_version != ixgbe_mbox_api_16) + return IXGBE_ERR_FEATURE_NOT_SUPPORTED; + + msgbuf[0] = IXGBE_VF_GET_PF_LINK_STATE; + err = ixgbevf_write_msg_read_ack(hw, msgbuf, msgbuf, 3); + if (err) + return (err); + if (msgbuf[0] & IXGBE_VT_MSGTYPE_FAILURE) { + *speed = IXGBE_LINK_SPEED_UNKNOWN; + return (IXGBE_ERR_MBX); + } + if ((msgbuf[0] & IXGBE_VT_MSGTYPE_CTS) == 0 || + (msgbuf[0] & ~IXGBE_VT_MSGTYPE_CTS) != + (IXGBE_VF_GET_PF_LINK_STATE | IXGBE_VT_MSGTYPE_SUCCESS)) { + *speed = IXGBE_LINK_SPEED_UNKNOWN; + return (IXGBE_ERR_MBX); + } + + *speed = msgbuf[1]; + *link_up = msgbuf[2]; + + return err; +} + /** * ixgbe_set_vfta_vf - Set/Unset vlan filter table address * @hw: pointer to the HW structure @@ -623,12 +665,29 @@ s32 ixgbe_check_mac_link_vf(struct ixgbe_hw *hw, ixgbe_link_speed *speed, UNREFERENCED_1PARAMETER(autoneg_wait_to_complete); + if (link_up == NULL) + return (IXGBE_ERR_PARAM); + /* If we were hit with a reset drop the link */ if (!mbx->ops[0].check_for_rst(hw, 0) || !mbx->timeout) mac->get_link_status = true; if (!mac->get_link_status) goto out; + if (mac->type == ixgbe_mac_E610_vf && + hw->api_version == ixgbe_mbox_api_16) { + ret_val = ixgbevf_get_pf_link_state(hw, speed, link_up); + if (ret_val == IXGBE_ERR_MBX) { + /* A PF NACK means the VF has lost clear-to-send state. */ + *link_up = false; + return (ret_val); + } else if (ret_val != IXGBE_SUCCESS) { + /* Let the driver debounce transient transport failures. */ + return (ret_val); + } + mac->get_link_status = !*link_up; + return (IXGBE_SUCCESS); + } /* if link status is down no point in checking to see if pf is up */ links_reg = IXGBE_READ_REG(hw, IXGBE_VFLINKS); @@ -777,6 +836,7 @@ int ixgbevf_get_queues(struct ixgbe_hw *hw, unsigned int *num_tcs, case ixgbe_mbox_api_12: case ixgbe_mbox_api_13: case ixgbe_mbox_api_15: + case ixgbe_mbox_api_16: break; default: return 0; diff --git a/sys/dev/ixgbe/ixgbe_vf.h b/sys/dev/ixgbe/ixgbe_vf.h index 2e8921a21fed..7f5a8c2e85cf 100644 --- a/sys/dev/ixgbe/ixgbe_vf.h +++ b/sys/dev/ixgbe/ixgbe_vf.h @@ -135,6 +135,8 @@ s32 ixgbe_update_mc_addr_list_vf(struct ixgbe_hw *hw, u8 *mc_addr_list, bool clear); s32 ixgbevf_update_xcast_mode(struct ixgbe_hw *hw, int xcast_mode); s32 ixgbe_get_link_state_vf(struct ixgbe_hw *hw, bool *link_state); +int ixgbevf_get_pf_link_state(struct ixgbe_hw *hw, ixgbe_link_speed *speed, + bool *link_up); s32 ixgbe_set_vfta_vf(struct ixgbe_hw *hw, u32 vlan, u32 vind, bool vlan_on, bool vlvf_bypass); s32 ixgbevf_rlpml_set_vf(struct ixgbe_hw *hw, u16 max_size);