From nobody Fri Sep 18 02:15:00 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hmGRT1GDGz6sS2G for ; Fri, 18 Sep 2026 02:15:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hmGRT10cRz3K9t for ; Fri, 18 Sep 2026 02:15:01 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1789697701; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=nOWTFN5lCx3Vbg3UShTL4ybLeSUCAi2PPQfzugiizsc=; b=FKqSOcrv/6A9I2bmh1kY+KbcE0yAsXAt+s2WKGMaQF8XDdKmBE4i2KHyDw9n4k6bk2FHk0 h9fk2aO8omSe5hsFTf42f0NPXJcOIx16zZhcw12ejxuIJDHxKt37+yALDFuNTKWRYZooDX V3NuKKR/awkmD6uFPASeXjz/4cZLwfmL/QkcpVFflxs6vsIVx5wn730HRO5AUdqcSaufJA 26ujTKqS8RZEYwaOudjLwowC4nsiOPnimQTmlr9ef7otRvWjbgoHjAiju7XbA+NPKe8EAr JPZBLZpHGygTKB4bo8ZieH0QdZc0jpDa4o8d0HQSbADRYKy6c26Wq2hE1FqWvQ== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1789697701; b=mZDmIvkoyl3BcUkI8/xjdxW65/dfFtJY5923ZKlPp1MWc7SMnOqsG8P8siwNcGftcS0XyZ VFpMLv5L9nEELadUO1fPlnFHvWH/sVq9+wS1cp/59mVj5Btad9zCYjxehLa7EbsviOli7I WDZpNMF0yOppa7k0aUxFp0+OcHOioIQli/N5FFbBREC/O6Wv0EAW3VvMDvXV6rJXa+rPJr EGy+dHtEC3KKHYsEq+LCLA1LdmQNuY7Z2ihsXvs2tnFHuxwJ1PhGvTnoLbB/ItCXvx7U+V cvAS9TjfgIjFQPgKCvPTD7HsNdURk95B6LeguQqUo5wdkQPUi7B6ZiAYbjbcMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1789697701; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=nOWTFN5lCx3Vbg3UShTL4ybLeSUCAi2PPQfzugiizsc=; b=UjaSDfpIuiJyxJraDfn7FsV+gyT05A83o3EEpVHtLPfh7izL9AwssOrCQMcetffu1/RmQs Re3mF2G9XcxVysbkgYc3YHwHIYzs3FSsvz3nEq//vpmAOUESyj7y8cfCdM/WXS+K0fYqyz 2ZGFc6tuXZLugaiWIrkLYA3y81wKI+vXuUukLR1Dth5yIP6x460yRgIP0CsRuYe2Cvqazr 1LOunNhGRmy52LeXPhnhQRUQjqkkyUbuPUdKq4U96MaFThpNQ4gKlF1HO1zsZdALFnd0P8 t87nPi5y3blptw2spVpKcWYoSmCPPFe7Fy3DzIGZoiqZwGPTSu1lBH0k+tvKEg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hmGRT04YHzyRr for ; Fri, 18 Sep 2026 02:15:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 3e76e by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Fri, 18 Sep 2026 02:15:00 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Kevin Bowling Subject: git: 31a77c113507 - main - ice: Report SR-IOV VF status List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kbowling X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 31a77c113507d5e7689996976679a972a11ff5f7 Auto-Submitted: auto-generated Date: Fri, 18 Sep 2026 02:15:00 +0000 Message-Id: <6aac9ea4.3e76e.2f2bf74e@gitrepo.freebsd.org> The branch main has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=31a77c113507d5e7689996976679a972a11ff5f7 commit 31a77c113507d5e7689996976679a972a11ff5f7 Author: Kevin Bowling AuthorDate: 2026-09-06 16:17:40 +0000 Commit: Kevin Bowling CommitDate: 2026-09-18 02:12:51 +0000 ice: Report SR-IOV VF status Report the VF MAC, allocated transmit and receive queues, exact trunk VLAN-filter count and capacity, negotiated virtchnl API, configured MAC, VLAN, spoof-check, and promiscuous-mode policy, automatic link-state policy, PF traffic permission, and fault containment through iflib. Expose mirror configuration and active hardware rules, precise malicious-driver isolation and counters, software mailbox-overflow isolation and counters, VF-owned MAC-filter count and limit, and reset diagnostics through a versioned driver.ice extension. Distinguish a failed VF reset from a required VSI rebuild, which may still be pending rather than failed. Keep the namespace schema local to the driver so future extensions need no changes to common network headers or the formatter. Invalidate cached VF handshakes during preparation for an externally initiated device reset, before releasing the context lock to wait for hardware. Mark the VFs as requiring rebuild even if an early PF rebuild failure prevents reaching their VSIs. Software-initiated resets retain their advance notification before invalidating the handshake. The iflib context lock protects VF state and VSI lifetime while constructing the snapshot. The query uses only cached state and does not issue AdminQ commands or read device registers. Validation on E810 with host-attached iavf VFs used ifconfig -v and direct Netlink queries to check handshake, queue, VLAN, policy, MDD, and mailbox state across VF and PF resets. Reviewed by: Pawel Sobczyk , ziaee Sponsored by: BBOX.io Differential Revision: https://reviews.freebsd.org/D58739 --- share/man/man4/ice.4 | 73 ++++++++++++++++- sys/dev/ice/ice_iov.c | 189 +++++++++++++++++++++++++++++++++++++++++++-- sys/dev/ice/ice_iov.h | 4 +- sys/dev/ice/if_ice_iflib.c | 30 +++++++ 4 files changed, 287 insertions(+), 9 deletions(-) diff --git a/share/man/man4/ice.4 b/share/man/man4/ice.4 index 91d8f95e8fc7..5110d62b5b49 100644 --- a/share/man/man4/ice.4 +++ b/share/man/man4/ice.4 @@ -32,7 +32,7 @@ .\" .\" * Other names and brands may be claimed as the property of others. .\" -.Dd September 5, 2026 +.Dd September 17, 2026 .Dt ICE 4 .Os .Sh NAME @@ -1195,6 +1195,76 @@ and discards its requests so that other VFs can continue to use the mailbox. An externally initiated VF function-level reset, PF reset, or SR-IOV configuration recreation releases the VF. .Pp +For each VF in an active SR-IOV configuration, +.Xr ifconfig 8 +with the +.Fl v +option displays cached PF state, including its MAC address, allocated transmit +and receive queues, exact trunk VLAN-filter count and capacity, negotiated +virtual-channel API, and configured MAC, VLAN, anti-spoof, and promiscuous-mode +policy. +The VF link-state policy is automatic: each VF follows the PF link state. +The API version is present only after the VF completes its resource handshake +following the last reset. +The generic traffic-permission and fault-containment fields indicate whether +PF policy currently permits the VF to issue traffic and whether MDD or +software mailbox protection has isolated it. +They do not indicate link state or observed traffic. +.Pp +Structured status consumers also receive mirroring, malicious-driver, +mailbox, MAC-filter, and reset diagnostics in the version 1 +.Cm driver.ice +namespace of the +.Xr rtnetlink 4 +VF status response: +.Bl -tag -width "mailbox-overflow-events" +.It Cm version +Namespace version number, currently 1. +.It Cm mirror-configured +Boolean indicating whether a mirror source VSI is configured. +.It Cm mirror-source-vsi +The source VSI number; omitted when no mirror is configured. +.It Cm mirror-ingress-active +Boolean indicating whether the ingress hardware mirror rule is installed. +.It Cm mirror-egress-active +Boolean indicating whether the egress hardware mirror rule is installed. +.It Cm mdd-blocked +Boolean indicating that malicious-driver detection has isolated the VF. +.It Cm mdd-tx-events +Cumulative number of transmit malicious-driver latches attributed to the VF. +.It Cm mdd-rx-events +Cumulative number of receive malicious-driver latches attributed to the VF. +.It Cm mailbox-blocked +Boolean indicating that software mailbox-overflow detection has isolated the +VF. +This field is omitted on E830 controllers, which enforce the mailbox limit in +hardware without persistent software isolation. +.It Cm mailbox-overflow-events +Cumulative number of times software mailbox-overflow detection has isolated +the VF. +This field is omitted on E830 controllers. +.It Cm mac-filter-count +Number of VF-owned explicit MAC filters recorded by the PF, including unicast +and multicast addresses. +The administrator-assigned MAC and implicit broadcast filter do not count +toward this quota. +.It Cm mac-filter-limit +Maximum number of VF-owned explicit MAC filters permitted by PF policy. +This field is omitted until the PF has accepted the VF configuration. +.It Cm reset-failed +Boolean indicating that a VF reset or restoration of its PF-owned policy +failed. +The VF remains held until reset recovery succeeds. +.It Cm rebuild-required +Boolean indicating that the VF's PF-owned VSI state needs rebuilding. +This includes a pending rebuild during PF or device reset as well as a failed +rebuild; it does not by itself indicate failure. +A VF reset cannot recover this state. +.El +.Pp +The status query uses driver-cached state and does not issue AdminQ commands +or read device registers. +.Pp An up to date list of parameters and their defaults can be found by using .Xr iovctl 8 with the @@ -1252,6 +1322,7 @@ email all the specific information related to the issue to .Sh SEE ALSO .Xr iflib 4 , .Xr led 4 , +.Xr rtnetlink 4 , .Xr vlan 4 , .Xr ifconfig 8 , .Xr sysctl 8 , diff --git a/sys/dev/ice/ice_iov.c b/sys/dev/ice/ice_iov.c index d60b40b0e24d..e5ee1a03936d 100644 --- a/sys/dev/ice/ice_iov.c +++ b/sys/dev/ice/ice_iov.c @@ -40,6 +40,43 @@ #include "ice_iov.h" #include "ice_fault.h" +#include + +/* Version 1 driver.ice extension schema; documented in ice(4). */ +#define ICE_VF_STATUS_NAMESPACE "driver.ice" +#define ICE_VF_STATUS_VERSION 1 +#define ICE_VF_STATUS_MIRROR_CONFIGURED "mirror-configured" +#define ICE_VF_STATUS_MIRROR_SOURCE_VSI "mirror-source-vsi" +#define ICE_VF_STATUS_MIRROR_INGRESS_ACTIVE "mirror-ingress-active" +#define ICE_VF_STATUS_MIRROR_EGRESS_ACTIVE "mirror-egress-active" +#define ICE_VF_STATUS_MDD_BLOCKED "mdd-blocked" +#define ICE_VF_STATUS_MDD_TX_EVENTS "mdd-tx-events" +#define ICE_VF_STATUS_MDD_RX_EVENTS "mdd-rx-events" +#define ICE_VF_STATUS_MBX_BLOCKED "mailbox-blocked" +#define ICE_VF_STATUS_MBX_OVERFLOW_EVENTS "mailbox-overflow-events" +#define ICE_VF_STATUS_MAC_FILTER_COUNT "mac-filter-count" +#define ICE_VF_STATUS_MAC_FILTER_LIMIT "mac-filter-limit" +#define ICE_VF_STATUS_RESET_FAILED "reset-failed" +#define ICE_VF_STATUS_REBUILD_REQUIRED "rebuild-required" + +/* Optional fields are compacted when absent; values define schema order. */ +enum ice_vf_status_field { + ICE_VF_STATUS_FIELD_MIRROR_CONFIGURED, + ICE_VF_STATUS_FIELD_MIRROR_SOURCE_VSI, + ICE_VF_STATUS_FIELD_MIRROR_INGRESS_ACTIVE, + ICE_VF_STATUS_FIELD_MIRROR_EGRESS_ACTIVE, + ICE_VF_STATUS_FIELD_MDD_BLOCKED, + ICE_VF_STATUS_FIELD_MDD_TX_EVENTS, + ICE_VF_STATUS_FIELD_MDD_RX_EVENTS, + ICE_VF_STATUS_FIELD_MBX_BLOCKED, + ICE_VF_STATUS_FIELD_MBX_OVERFLOW_EVENTS, + ICE_VF_STATUS_FIELD_MAC_FILTER_COUNT, + ICE_VF_STATUS_FIELD_MAC_FILTER_LIMIT, + ICE_VF_STATUS_FIELD_RESET_FAILED, + ICE_VF_STATUS_FIELD_REBUILD_REQUIRED, + ICE_VF_STATUS_NUM_FIELDS, +}; + #define ICE_VC_MAX_RX_BUFFER \ ((16 * 1024) - BIT(ICE_RLAN_CTX_DBUF_S)) #define ICE_VIRTCHNL_QUEUE_MAP_SIZE 16 @@ -631,6 +668,144 @@ release_vsi: return (error); } +/** + * ice_iov_vf_status - report configured VF state + * @sc: device private structure + * @statusp: returned status snapshot + * + * The iflib context lock protects VF state and VSI lifetime while this + * method constructs the report. + */ +int +ice_iov_vf_status(struct ice_softc *sc, struct if_vf_status **statusp) +{ + struct ice_vf *vf; + struct ice_vsi *vsi; + struct if_vf_extension *extension; + struct if_vf_info *info; + struct if_vf_status *status; + u32 vf_flags; + bool mirror_configured, software_mbx_limit; + uint32_t field, num_fields; + int i; + + if (!ice_is_bit_set(sc->feat_en, ICE_FEATURE_SRIOV)) + return (EOPNOTSUPP); + status = if_vf_status_alloc(sc->num_vfs); + if (status == NULL) + return (ENOMEM); + for (i = 0; i < sc->num_vfs; i++) { + vf = &sc->vfs[i]; + vsi = vf->vsi; + vf_flags = atomic_load_acq_32(&vf->vf_flags); + info = &status->vfs[i]; + info->fields = IFVF_F_CONFIGURED | IFVF_F_INITIALIZED | + IFVF_F_TRAFFIC_ALLOWED | IFVF_F_FAULT_BLOCKED | + IFVF_F_LINK_STATE_POLICY | + IFVF_F_VLAN_MODE | IFVF_F_VLAN_COUNT | + IFVF_F_ALLOW_SET_MAC | IFVF_F_ALLOW_SET_VLAN | + IFVF_F_MAC_ANTI_SPOOF | IFVF_F_ALLOW_PROMISC; + info->index = i; + info->configured = + (vf_flags & VF_FLAG_ENABLED) != 0 && vsi != NULL; + info->initialized = info->configured && + (vf_flags & VF_FLAG_INITIALIZED) != 0; + info->traffic_allowed = info->configured && + (vf_flags & (VF_FLAG_MDD_BLOCKED | + VF_FLAG_MBX_BLOCKED)) == 0; + info->fault_blocked = (vf_flags & (VF_FLAG_MDD_BLOCKED | + VF_FLAG_MBX_BLOCKED)) != 0; + info->link_state_policy = IFVF_LINK_AUTO; + if (info->initialized) { + snprintf(info->api_version, sizeof(info->api_version), + "%u.%u", vf->version.major, vf->version.minor); + info->fields |= IFVF_F_API_VERSION; + } + if (!ETHER_IS_ZERO(vf->mac)) { + memcpy(info->mac, vf->mac, sizeof(info->mac)); + info->fields |= IFVF_F_MAC; + } + /* The ICE IOV schema exposes only VF-managed trunk membership. */ + info->vlan_mode = IFVF_VLAN_TRUNK; + info->vlan_count = vf->vlan_cnt; + if (info->configured) { + info->vlan_limit = vf->vlan_limit; + info->fields |= IFVF_F_VLAN_LIMIT; + } + if (vsi != NULL) { + info->tx_queue_count = vsi->num_tx_queues; + info->rx_queue_count = vsi->num_rx_queues; + info->fields |= IFVF_F_NUM_TX_QUEUES | + IFVF_F_NUM_RX_QUEUES; + } + + mirror_configured = vsi != NULL && vsi->mirror_src_vsi != + ICE_INVALID_MIRROR_VSI; + software_mbx_limit = !ice_is_e830(&sc->hw); + num_fields = ICE_VF_STATUS_NUM_FIELDS - + (mirror_configured ? 0 : 1) - + (software_mbx_limit ? 0 : 2) - + (info->configured ? 0 : 1); + extension = if_vf_status_add_extension(info, + ICE_VF_STATUS_NAMESPACE, ICE_VF_STATUS_VERSION, + num_fields); + if (extension == NULL) { + if_vf_status_free(status); + return (ENOMEM); + } + field = ICE_VF_STATUS_FIELD_MIRROR_CONFIGURED; + if_vf_extension_set_bool(extension, field++, + ICE_VF_STATUS_MIRROR_CONFIGURED, mirror_configured); + if (mirror_configured) + if_vf_extension_set_number(extension, field++, + ICE_VF_STATUS_MIRROR_SOURCE_VSI, + vsi->mirror_src_vsi); + if_vf_extension_set_bool(extension, field++, + ICE_VF_STATUS_MIRROR_INGRESS_ACTIVE, + vsi != NULL && + vsi->rule_mir_ingress != ICE_INVAL_MIRROR_RULE_ID); + if_vf_extension_set_bool(extension, field++, + ICE_VF_STATUS_MIRROR_EGRESS_ACTIVE, + vsi != NULL && + vsi->rule_mir_egress != ICE_INVAL_MIRROR_RULE_ID); + if_vf_extension_set_bool(extension, field++, + ICE_VF_STATUS_MDD_BLOCKED, + (vf_flags & VF_FLAG_MDD_BLOCKED) != 0); + if_vf_extension_set_number(extension, field++, + ICE_VF_STATUS_MDD_TX_EVENTS, vf->mdd_tx_events); + if_vf_extension_set_number(extension, field++, + ICE_VF_STATUS_MDD_RX_EVENTS, vf->mdd_rx_events); + if (software_mbx_limit) { + if_vf_extension_set_bool(extension, field++, + ICE_VF_STATUS_MBX_BLOCKED, + (vf_flags & VF_FLAG_MBX_BLOCKED) != 0); + if_vf_extension_set_number(extension, field++, + ICE_VF_STATUS_MBX_OVERFLOW_EVENTS, + vf->mbx_overflow_events); + } + if_vf_extension_set_number(extension, field++, + ICE_VF_STATUS_MAC_FILTER_COUNT, vf->mac_filter_cnt); + if (info->configured) + if_vf_extension_set_number(extension, field++, + ICE_VF_STATUS_MAC_FILTER_LIMIT, vf->mac_filter_limit); + if_vf_extension_set_bool(extension, field++, + ICE_VF_STATUS_RESET_FAILED, + (vf_flags & VF_FLAG_RESET_FAILED) != 0); + if_vf_extension_set_bool(extension, field++, + ICE_VF_STATUS_REBUILD_REQUIRED, + (vf_flags & VF_FLAG_REBUILD_REQUIRED) != 0); + KASSERT(field == num_fields, + ("ICE VF status field count %u != %u", field, num_fields)); + info->allow_set_mac = (vf_flags & VF_FLAG_SET_MAC_CAP) != 0; + info->allow_set_vlan = (vf_flags & VF_FLAG_VLAN_CAP) != 0; + info->mac_anti_spoof = + (vf_flags & VF_FLAG_MAC_ANTI_SPOOF) != 0; + info->allow_promisc = (vf_flags & VF_FLAG_PROMISC_CAP) != 0; + } + *statusp = status; + return (0); +} + /** * ice_iov_uninit - Called by the OS when VFs are destroyed * @sc: device softc structure @@ -722,7 +897,7 @@ ice_iov_handle_vflr(struct ice_softc *sc) continue; vf_flags = atomic_load_acq_32(&vf->vf_flags); if ((vf_flags & VF_FLAG_ENABLED) != 0 && vf->vsi != NULL) { - if ((vf_flags & VF_FLAG_REBUILD_FAILED) != 0) { + if ((vf_flags & VF_FLAG_REBUILD_REQUIRED) != 0) { /* Consume the event but leave the invalid VF held. */ wr32(hw, GLGEN_VFLRSTAT(reg_idx), BIT(bit_idx)); ice_flush(hw); @@ -996,7 +1171,7 @@ ice_iov_rebuild_vf(struct ice_softc *sc, struct ice_vsi *vsi) MPASS(vsi->type == ICE_VSI_VF); vf = ice_iov_get_vf(sc, vsi->vf_num); atomic_clear_32(&vf->vf_flags, VF_FLAG_INITIALIZED); - atomic_set_32(&vf->vf_flags, VF_FLAG_REBUILD_FAILED); + atomic_set_32(&vf->vf_flags, VF_FLAG_REBUILD_REQUIRED); ice_iov_clear_vf_queue_state(vf); ICE_IOV_FAIL_POINT(sc, vf->vf_num, rebuild_before_initialize, error, fail); @@ -1025,7 +1200,7 @@ ice_iov_rebuild_vf(struct ice_softc *sc, struct ice_vsi *vsi) } atomic_clear_32(&vf->vf_flags, - VF_FLAG_REBUILD_FAILED | VF_FLAG_RESET_FAILED); + VF_FLAG_REBUILD_REQUIRED | VF_FLAG_RESET_FAILED); ice_iov_ready_vf(sc, vf); return (0); @@ -1064,7 +1239,7 @@ ice_reset_vf(struct ice_softc *sc, struct ice_vf *vf, bool trigger_reset, /* A VFR cannot recover PF-owned VSI state lost during PF rebuild. */ if (release_vf && (atomic_load_acq_32(&vf->vf_flags) & - VF_FLAG_REBUILD_FAILED) != 0) + VF_FLAG_REBUILD_REQUIRED) != 0) return (EIO); global_vf_num = vf->vf_num + hw->func_caps.vf_base_id; @@ -1234,7 +1409,7 @@ ice_iov_quiesce_vfs_for_reset(struct ice_softc *sc) /* Block mailbox reconfiguration before asserting reset. */ atomic_clear_32(&vf->vf_flags, VF_FLAG_INITIALIZED); - atomic_set_32(&vf->vf_flags, VF_FLAG_REBUILD_FAILED); + atomic_set_32(&vf->vf_flags, VF_FLAG_REBUILD_REQUIRED); reg = rd32(hw, VPGEN_VFRTRIG(vf->vf_num)); reg |= VPGEN_VFRTRIG_VFSWR_M; wr32(hw, VPGEN_VFRTRIG(vf->vf_num), reg); @@ -2889,9 +3064,9 @@ ice_vc_handle_vf_msg(struct ice_softc *sc, struct ice_rq_event_info *event, /* * Permit only reset negotiation while VF hardware state is unsafe. - * A VFR can retry RESET_FAILED; REBUILD_FAILED requires a PF rebuild. + * A VFR can retry RESET_FAILED; REBUILD_REQUIRED needs a PF rebuild. */ - if ((vf_flags & (VF_FLAG_REBUILD_FAILED | VF_FLAG_RESET_FAILED)) != 0 && + if ((vf_flags & (VF_FLAG_REBUILD_REQUIRED | VF_FLAG_RESET_FAILED)) != 0 && v_opcode != VIRTCHNL_OP_VERSION && v_opcode != VIRTCHNL_OP_RESET_VF) { ice_aq_send_msg_to_vf(hw, v_id, v_opcode, diff --git a/sys/dev/ice/ice_iov.h b/sys/dev/ice/ice_iov.h index 3ef565f709cc..bdb28b4f7e92 100644 --- a/sys/dev/ice/ice_iov.h +++ b/sys/dev/ice/ice_iov.h @@ -71,7 +71,7 @@ enum ice_vf_flags { VF_FLAG_PROMISC_CAP = BIT(3), VF_FLAG_MAC_ANTI_SPOOF = BIT(4), VF_FLAG_INITIALIZED = BIT(5), - VF_FLAG_REBUILD_FAILED = BIT(6), + VF_FLAG_REBUILD_REQUIRED = BIT(6), VF_FLAG_RESET_FAILED = BIT(7), VF_FLAG_MDD_BLOCKED = BIT(8), VF_FLAG_MBX_BLOCKED = BIT(9), @@ -139,6 +139,8 @@ int ice_iov_detach(struct ice_softc *sc); int ice_iov_init(struct ice_softc *sc, uint16_t num_vfs, const nvlist_t *params); int ice_iov_add_vf(struct ice_softc *sc, uint16_t vfnum, const nvlist_t *params); int ice_iov_rebuild_vf(struct ice_softc *sc, struct ice_vsi *vsi); +struct if_vf_status; +int ice_iov_vf_status(struct ice_softc *sc, struct if_vf_status **statusp); void ice_iov_uninit(struct ice_softc *sc); void ice_iov_handle_vflr(struct ice_softc *sc); diff --git a/sys/dev/ice/if_ice_iflib.c b/sys/dev/ice/if_ice_iflib.c index 9068fd385314..d9143618a0ec 100644 --- a/sys/dev/ice/if_ice_iflib.c +++ b/sys/dev/ice/if_ice_iflib.c @@ -93,6 +93,7 @@ static void ice_init_link(struct ice_softc *sc); static int ice_if_iov_init(if_ctx_t ctx, uint16_t num_vfs, const nvlist_t *params); static void ice_if_iov_uninit(if_ctx_t ctx); static int ice_if_iov_vf_add(if_ctx_t ctx, uint16_t vfnum, const nvlist_t *params); +static int ice_if_vf_status(if_ctx_t ctx, struct if_vf_status **statusp); static void ice_if_vflr_handle(if_ctx_t ctx); #endif static int ice_setup_mirror_vsi(struct ice_mirr_if *mif); @@ -220,6 +221,7 @@ static device_method_t ice_iflib_methods[] = { DEVMETHOD(ifdi_iov_vf_add, ice_if_iov_vf_add), DEVMETHOD(ifdi_iov_init, ice_if_iov_init), DEVMETHOD(ifdi_iov_uninit, ice_if_iov_uninit), + DEVMETHOD(ifdi_vf_status, ice_if_vf_status), DEVMETHOD(ifdi_vflr_handle, ice_if_vflr_handle), #endif DEVMETHOD_END @@ -2613,6 +2615,21 @@ ice_prepare_for_reset(struct ice_softc *sc) if (error != 0) device_printf(sc->dev, "Failed to quiesce one or more VFs: %d\n", error); + } else { + /* + * Hardware has already gated the VFs. Invalidate their cached + * handshake before dropping CTX_LOCK to wait for reset, even if + * rebuilding later fails before reaching the VF VSIs. + */ + for (int i = 0; i < sc->num_vfs; i++) { + struct ice_vf *vf = &sc->vfs[i]; + + if ((atomic_load_acq_32(&vf->vf_flags) & + VF_FLAG_ENABLED) == 0 || vf->vsi == NULL) + continue; + atomic_clear_32(&vf->vf_flags, VF_FLAG_INITIALIZED); + atomic_set_32(&vf->vf_flags, VF_FLAG_REBUILD_REQUIRED); + } } #endif @@ -3635,6 +3652,19 @@ ice_if_iov_vf_add(if_ctx_t ctx, uint16_t vfnum, const nvlist_t *params) return ice_iov_add_vf(sc, vfnum, params); } +/** + * ice_if_vf_status - report configured VF state + * @ctx: iflib context pointer + * @statusp: returned VF status snapshot + */ +static int +ice_if_vf_status(if_ctx_t ctx, struct if_vf_status **statusp) +{ + struct ice_softc *sc = (struct ice_softc *)iflib_get_softc(ctx); + + return (ice_iov_vf_status(sc, statusp)); +} + /** * ice_if_vflr_handle - iov VFLR handler * @ctx: iflib context pointer