From nobody Mon Aug 10 02:32:46 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4hJJh35Xv9z6pVZq for ; Mon, 10 Aug 2026 02:32:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4hJJh33bNBz42lw for ; Mon, 10 Aug 2026 02:32:51 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786329171; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=jXyFXZgbEZlfgdhXBmoexyA+5TVxym03oROXNREPhSc=; b=uHuxhg+CkKRIVDWPuEj/lb7phDvkStx42JOow+wPpGTg+7kGp/uHIatXHWVOn0e76atqNJ /35+WyWYOc4OhAtvNpCGpFo8ZFxXZCMJrpIjZh2avNebSLDVEd0x9yHJltkdnQXAF4GzS2 sSK62ySdHt1mZ0Kyz//K9USeSKUpejDxM3+2Xgf/kEfC923slbZ932GDhMCWCvh7kehIUJ WVvnA0tZpVEjCxdPhdm1PDm1Vsg2Iu/wACqfpAFe4MCFIVNZ0XNtj9hYqLJ5ABwkblvRQ1 naeWzsiCAy38h9pTZFt8HGSpg7cRaTDhxzifOenK+dEv+QCnS/Cg6TByJmKvyw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1786329171; a=rsa-sha256; cv=none; b=nFs4kZyJvLF3LO3jsbHgCi9J40YsJPV+Y77x37P0sA8X13FFU6YcEBoT/8Z09+zPpl+lzZ VMr7LR+RJ0zJxkQLDWLYzPTUj1B7fP1CA81cUDIhHV5ROxdeEtSAWuV1X9dxiR0V2TcstY vUYFkfhcCfaseYY042s9C5HS9nou/MWDy5YuXYuQhxy338m5XTz+9P2joRmQveNnlqIOyy yaBZ7L9FjVKOLiyNw4UqyXpF6n9GTPiYlx/Rg+eXw2MtSsDC7wu6RQtvTLKa94qVIghmOz wMfT3b/jq4GQCxobrXKAvlncFE08znWnt0jQqIFSW2eQiq9sHyqQItpqdQiR6A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1786329171; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=jXyFXZgbEZlfgdhXBmoexyA+5TVxym03oROXNREPhSc=; b=I37Nki28uATFoRBAHtdUXYAcBD/NoMbo++9E3BeqY+NYGCcLTieh5uz9FUYTNGaZPcZx+G vWma1cI58rGY9etOVLAhP8wtQcnzabJUii1GfYBRsrkj2yl0YOyWOQhfeWWxI8S+MM+GWT pAzExyWCgJkUPR8h62Jxxz+sn6x7/uhnfzY+pvM/X/HtArKkT5c5EjusvmoM+l9eV0EDpA U1blP8O2MKB1VGg3/GbCWdKb68BXBKh9Hl53mxK9lcYRU9fs7+9UzgkbYxRAJxpF0MBNVs 5wprebyf93TsH0ugcUfXXk/eYlAwmcWFbmD4/BZ/CQoqa28CkgBvKd2e87VTvA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4hJJh32RN5zq9q for ; Mon, 10 Aug 2026 02:32:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 1f5db by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Mon, 10 Aug 2026 02:32:46 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Kevin Bowling Subject: git: e9a4d3969a31 - main - iavf: Recover when PF communication is unavailable List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kbowling X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: e9a4d3969a3164d41d41480bacd3520fcf05ccbf Auto-Submitted: auto-generated Date: Mon, 10 Aug 2026 02:32:46 +0000 Message-Id: <6a79384e.1f5db.547debf8@gitrepo.freebsd.org> The branch main has been updated by kbowling: URL: https://cgit.FreeBSD.org/src/commit/?id=e9a4d3969a3164d41d41480bacd3520fcf05ccbf commit e9a4d3969a3164d41d41480bacd3520fcf05ccbf Author: Kevin Bowling AuthorDate: 2026-08-09 09:36:12 +0000 Commit: Kevin Bowling CommitDate: 2026-08-10 02:32:40 +0000 iavf: Recover when PF communication is unavailable A PF reset or loss of virtchnl service can make visible interface initialization wait up to ten seconds and then return from the void ifdi_init callback. Iflib consequently marks the interface running even though its queues were not initialized, and no retry is scheduled when the PF returns. Check reset readiness without polling during reinitialization, propagate queue-message submission errors, and bound a silent enable or disable to one mailbox timeout. Report unsuccessful initialization to iflib and publish link-down state without polling the stopped mailbox. A VFLR also discards the Admin Queue and permits the PF to replace the VF VSI. Track when full virtchnl rediscovery is required, renegotiate the API version, refresh and validate the VF resources before using a cached VSI ID, and replay the MAC and VLAN filters cleared by reset. Bound each runtime discovery attempt while preserving the existing attach-time wait. While the VF remains administratively up, retry complete initialization after 250 ms, one second, four seconds, and then at a capped eight-second interval. MFC after: 2 weeks --- sys/dev/iavf/iavf_iflib.h | 8 + sys/dev/iavf/iavf_lib.c | 48 +++-- sys/dev/iavf/iavf_lib.h | 4 +- sys/dev/iavf/iavf_vc_common.c | 73 +++++--- sys/dev/iavf/iavf_vc_common.h | 2 + sys/dev/iavf/if_iavf_iflib.c | 403 ++++++++++++++++++++++++++++++++++++++---- 6 files changed, 467 insertions(+), 71 deletions(-) diff --git a/sys/dev/iavf/iavf_iflib.h b/sys/dev/iavf/iavf_iflib.h index 83891f9ed520..b9289d8d4050 100644 --- a/sys/dev/iavf/iavf_iflib.h +++ b/sys/dev/iavf/iavf_iflib.h @@ -44,6 +44,7 @@ #include #include #include +#include #include #include #include @@ -375,6 +376,13 @@ struct iavf_sc { u32 queues_enabled; u8 enable_queues_chan; u8 disable_queues_chan; + u32 mbx_ready; + u32 vc_reinit_required; + u32 mbx_retry_pending; + struct callout mbx_retry; + struct timeval mbx_last_log; + u8 mbx_retry_stage; + bool mbx_retry_initialized; /* For virtchnl message processing task */ struct task vc_task; diff --git a/sys/dev/iavf/iavf_lib.c b/sys/dev/iavf/iavf_lib.c index 8596cf71bfff..e7cd9222bd68 100644 --- a/sys/dev/iavf/iavf_lib.c +++ b/sys/dev/iavf/iavf_lib.c @@ -183,6 +183,23 @@ iavf_sysctl_current_speed(SYSCTL_HANDLER_ARGS) return (error); } +/** + * iavf_reset_is_complete - Check whether a device reset is complete + * @hw: pointer to the hardware structure + * + * @returns true when reset is complete, or false otherwise. + */ +bool +iavf_reset_is_complete(struct iavf_hw *hw) +{ + u32 reg; + + reg = rd32(hw, IAVF_VFGEN_RSTAT) & + IAVF_VFGEN_RSTAT_VFR_STATE_MASK; + return (reg == VIRTCHNL_VFR_VFACTIVE || + reg == VIRTCHNL_VFR_COMPLETED); +} + /** * iavf_reset_complete - Wait for a device reset to complete * @hw: pointer to the hardware structure @@ -198,15 +215,10 @@ iavf_sysctl_current_speed(SYSCTL_HANDLER_ARGS) int iavf_reset_complete(struct iavf_hw *hw) { - u32 reg; /* Wait up to ~10 seconds */ for (int i = 0; i < 100; i++) { - reg = rd32(hw, IAVF_VFGEN_RSTAT) & - IAVF_VFGEN_RSTAT_VFR_STATE_MASK; - - if ((reg == VIRTCHNL_VFR_VFACTIVE) || - (reg == VIRTCHNL_VFR_COMPLETED)) + if (iavf_reset_is_complete(hw)) return (0); iavf_msec_pause(100); } @@ -1460,13 +1472,16 @@ iavf_mark_del_vlan_filter(struct iavf_sc *sc, u16 vtag) * Send a virtual channel message to the PF to DISABLE_QUEUES, but resend it up * to IAVF_MAX_DIS_Q_RETRY times if the response says that it wasn't * successful. This is intended to workaround a bug that can appear on the PF. + * + * @returns zero on success, or an error code if the request could not be sent + * or acknowledged. */ -void +int iavf_disable_queues_with_retries(struct iavf_sc *sc) { bool in_detach = iavf_driver_is_detaching(sc); int max_attempts = IAVF_MAX_DIS_Q_RETRY; - int msg_count = 0; + int error = 0, msg_count = 0; /* While the driver is detaching, it doesn't care if the queue * disable finishes successfully or not. Just send one message @@ -1478,7 +1493,10 @@ iavf_disable_queues_with_retries(struct iavf_sc *sc) while ((msg_count < max_attempts) && atomic_load_acq_32(&sc->queues_enabled)) { msg_count++; - iavf_send_vc_msg_sleep(sc, IAVF_FLAG_AQ_DISABLE_QUEUES); + error = iavf_send_vc_msg_sleep(sc, + IAVF_FLAG_AQ_DISABLE_QUEUES); + if (error != 0) + break; } /* Possibly print messages about retry attempts and issues */ @@ -1486,7 +1504,13 @@ iavf_disable_queues_with_retries(struct iavf_sc *sc) iavf_dbg_vc(sc, "DISABLE_QUEUES messages sent: %d\n", msg_count); - if (!in_detach && msg_count >= max_attempts) - device_printf(sc->dev, "%s: DISABLE_QUEUES may have failed\n", - __func__); + if (!in_detach && msg_count >= max_attempts && + atomic_load_acq_32(&sc->queues_enabled)) { + if (iavf_mbx_log_allowed(sc)) + device_printf(sc->dev, + "%s: DISABLE_QUEUES may have failed\n", __func__); + if (error == 0) + error = EIO; + } + return (error); } diff --git a/sys/dev/iavf/iavf_lib.h b/sys/dev/iavf/iavf_lib.h index 955f5c69288b..a483c9e4f755 100644 --- a/sys/dev/iavf/iavf_lib.h +++ b/sys/dev/iavf/iavf_lib.h @@ -426,11 +426,13 @@ cmp_etheraddr(const u8 *ea1, const u8 *ea2) int iavf_send_vc_msg(struct iavf_sc *sc, u32 op); int iavf_send_vc_msg_sleep(struct iavf_sc *sc, u32 op); +bool iavf_mbx_log_allowed(struct iavf_sc *sc); void iavf_update_link_status(struct iavf_sc *); bool iavf_driver_is_detaching(struct iavf_sc *sc); void iavf_msec_pause(int msecs); void iavf_get_default_rss_key(u32 *key); int iavf_allocate_pci_resources_common(struct iavf_sc *sc); +bool iavf_reset_is_complete(struct iavf_hw *hw); int iavf_reset_complete(struct iavf_hw *hw); int iavf_setup_vc(struct iavf_sc *sc); int iavf_reset(struct iavf_sc *sc); @@ -472,7 +474,7 @@ struct iavf_mac_filter * u64 iavf_baudrate_from_link_speed(struct iavf_sc *sc); void iavf_add_vlan_filter(struct iavf_sc *sc, u16 vtag); int iavf_mark_del_vlan_filter(struct iavf_sc *sc, u16 vtag); -void iavf_disable_queues_with_retries(struct iavf_sc *); +int iavf_disable_queues_with_retries(struct iavf_sc *); int iavf_sysctl_current_speed(SYSCTL_HANDLER_ARGS); int iavf_sysctl_tx_itr(SYSCTL_HANDLER_ARGS); diff --git a/sys/dev/iavf/iavf_vc_common.c b/sys/dev/iavf/iavf_vc_common.c index 8f45fe044094..37b085f7ef08 100644 --- a/sys/dev/iavf/iavf_vc_common.c +++ b/sys/dev/iavf/iavf_vc_common.c @@ -74,7 +74,7 @@ iavf_send_pf_msg(struct iavf_sc *sc, if (op != VIRTCHNL_OP_GET_STATS) device_printf(dev, "Unable to send opcode %s to PF, " "ASQ is not alive\n", iavf_vc_opcode_str(op)); - return (0); + return (IAVF_ERR_ADMIN_QUEUE_ERROR); } if (op != VIRTCHNL_OP_GET_STATS) @@ -122,11 +122,32 @@ iavf_send_api_ver(struct iavf_sc *sc) * Compare API versions with the PF. Must be called after admin queue is * initialized. * - * @returns 0 if API versions match, EIO if they do not, or - * IAVF_ERR_ADMIN_QUEUE_NO_WORK if the admin queue is empty. + * @returns 0 if API versions match, ETIMEDOUT if the PF does not reply, + * or EIO if the versions do not match. */ int iavf_verify_api_ver(struct iavf_sc *sc) +{ + int error; + + error = iavf_verify_api_ver_retries(sc, IAVF_AQ_MAX_ERR); + if (error == 0) + device_printf(sc->dev, "PF API %d.%d / VF API %d.%d\n", + sc->version.major, sc->version.minor, + VIRTCHNL_VERSION_MAJOR, VIRTCHNL_VERSION_MINOR); + return (error); +} + +/** + * iavf_verify_api_ver_retries - Verify the PF API version with a retry limit + * @sc: device softc + * @max_retries: maximum number of Admin Receive Queue polls + * + * @returns 0 if API versions match, ETIMEDOUT if the PF does not reply, + * or EIO for an invalid response or API version. + */ +int +iavf_verify_api_ver_retries(struct iavf_sc *sc, u32 max_retries) { struct virtchnl_version_info *pf_vvi; struct iavf_hw *hw = &sc->hw; @@ -134,14 +155,16 @@ iavf_verify_api_ver(struct iavf_sc *sc) enum iavf_status status; device_t dev = sc->dev; int error = 0; - int retries = 0; + u32 retries = 0; event.buf_len = IAVF_AQ_BUF_SZ; event.msg_buf = (u8 *)malloc(event.buf_len, M_IAVF, M_WAITOK); for (;;) { - if (++retries > IAVF_AQ_MAX_ERR) + if (++retries > max_retries) { + error = ETIMEDOUT; goto out_alloc; + } /* Initial delay here is necessary */ iavf_msec_pause(100); @@ -180,11 +203,6 @@ iavf_verify_api_ver(struct iavf_sc *sc) sc->version.minor = pf_vvi->minor; } - /* Log PF/VF api versions */ - device_printf(dev, "PF API %d.%d / VF API %d.%d\n", - pf_vvi->major, pf_vvi->minor, - VIRTCHNL_VERSION_MAJOR, VIRTCHNL_VERSION_MINOR); - out_alloc: free(event.msg_buf, M_IAVF); return (error); @@ -235,6 +253,21 @@ iavf_send_vf_config_msg(struct iavf_sc *sc) */ int iavf_get_vf_config(struct iavf_sc *sc) +{ + + return (iavf_get_vf_config_retries(sc, IAVF_AQ_MAX_ERR)); +} + +/** + * iavf_get_vf_config_retries - Get VF configuration with a retry limit + * @sc: device softc + * @max_retries: maximum number of Admin Receive Queue polls + * + * @returns zero on success, ETIMEDOUT if the PF does not reply, or an error + * for an invalid response. + */ +int +iavf_get_vf_config_retries(struct iavf_sc *sc, u32 max_retries) { struct iavf_hw *hw = &sc->hw; device_t dev = sc->dev; @@ -253,7 +286,7 @@ iavf_get_vf_config(struct iavf_sc *sc) for (;;) { status = iavf_clean_arq_element(hw, &event, NULL); if (status == IAVF_ERR_ADMIN_QUEUE_NO_WORK) { - if (++retries <= IAVF_AQ_MAX_ERR) + if (++retries <= max_retries) iavf_msec_pause(10); } else if ((enum virtchnl_ops)le32toh(event.desc.cookie_high) != VIRTCHNL_OP_GET_VF_RESOURCES) { @@ -278,7 +311,7 @@ iavf_get_vf_config(struct iavf_sc *sc) break; } - if (retries > IAVF_AQ_MAX_ERR) { + if (retries > max_retries) { iavf_dbg_vc(sc, "%s: Did not receive response after %d tries.", __func__, retries); @@ -287,6 +320,8 @@ iavf_get_vf_config(struct iavf_sc *sc) } } + bzero(sc->vf_res, sizeof(struct virtchnl_vf_resource) + + IAVF_MAX_VF_VSI * sizeof(struct virtchnl_vsi_resource)); memcpy(sc->vf_res, event.msg_buf, min(event.msg_len, len)); iavf_vf_parse_hw_config(hw, sc->vf_res); @@ -303,7 +338,7 @@ out_alloc: * * @remark the reply from the PF is not checked by this function. * - * @returns zero + * @returns zero on success, or an error code on failure. */ int iavf_enable_queues(struct iavf_sc *sc) @@ -314,9 +349,8 @@ iavf_enable_queues(struct iavf_sc *sc) vqs.vsi_id = sc->vsi_res->vsi_id; vqs.tx_queues = (1 << IAVF_NTXQS(vsi)) - 1; vqs.rx_queues = vqs.tx_queues; - iavf_send_pf_msg(sc, VIRTCHNL_OP_ENABLE_QUEUES, - (u8 *)&vqs, sizeof(vqs)); - return (0); + return (iavf_send_pf_msg(sc, VIRTCHNL_OP_ENABLE_QUEUES, + (u8 *)&vqs, sizeof(vqs))); } /** @@ -327,7 +361,7 @@ iavf_enable_queues(struct iavf_sc *sc) * * @remark the reply from the PF is not checked by this function. * - * @returns zero + * @returns zero on success, or an error code on failure. */ int iavf_disable_queues(struct iavf_sc *sc) @@ -338,9 +372,8 @@ iavf_disable_queues(struct iavf_sc *sc) vqs.vsi_id = sc->vsi_res->vsi_id; vqs.tx_queues = (1 << IAVF_NTXQS(vsi)) - 1; vqs.rx_queues = vqs.tx_queues; - iavf_send_pf_msg(sc, VIRTCHNL_OP_DISABLE_QUEUES, - (u8 *)&vqs, sizeof(vqs)); - return (0); + return (iavf_send_pf_msg(sc, VIRTCHNL_OP_DISABLE_QUEUES, + (u8 *)&vqs, sizeof(vqs))); } /** diff --git a/sys/dev/iavf/iavf_vc_common.h b/sys/dev/iavf/iavf_vc_common.h index 6acc3effcb3d..1b569148e0cc 100644 --- a/sys/dev/iavf/iavf_vc_common.h +++ b/sys/dev/iavf/iavf_vc_common.h @@ -44,12 +44,14 @@ int iavf_send_pf_msg(struct iavf_sc *sc, enum virtchnl_ops op, u8 *msg, u16 len); int iavf_verify_api_ver(struct iavf_sc *); +int iavf_verify_api_ver_retries(struct iavf_sc *, u32); int iavf_send_api_ver(struct iavf_sc *sc); int iavf_enable_queues(struct iavf_sc *sc); int iavf_disable_queues(struct iavf_sc *sc); int iavf_add_vlans(struct iavf_sc *sc); int iavf_send_vf_config_msg(struct iavf_sc *sc); int iavf_get_vf_config(struct iavf_sc *sc); +int iavf_get_vf_config_retries(struct iavf_sc *, u32); int iavf_del_vlans(struct iavf_sc *sc); int iavf_add_ether_filters(struct iavf_sc *sc); int iavf_del_ether_filters(struct iavf_sc *sc); diff --git a/sys/dev/iavf/if_iavf_iflib.c b/sys/dev/iavf/if_iavf_iflib.c index e4dd3b1e59a4..77c85f4212af 100644 --- a/sys/dev/iavf/if_iavf_iflib.c +++ b/sys/dev/iavf/if_iavf_iflib.c @@ -44,6 +44,19 @@ #include "iavf_drv_info.h" #include "iavf_sysctls_iflib.h" +static const sbintime_t iavf_mbx_retry_delay[] = { + 250 * SBT_1MS, + 1 * SBT_1S, + 4 * SBT_1S, + 8 * SBT_1S, +}; + +static const struct timeval iavf_mbx_log_interval = { 60, 0 }; + +#define IAVF_MBX_RECOVERY_ASQ_RETRIES 10 +#define IAVF_MBX_RECOVERY_VERSION_RETRIES 3 +#define IAVF_MBX_RECOVERY_CONFIG_RETRIES 10 + /********************************************************************* * Function prototypes *********************************************************************/ @@ -76,6 +89,16 @@ static void iavf_if_init(if_ctx_t ctx); static void iavf_if_stop(if_ctx_t ctx); static bool iavf_if_needs_restart(if_ctx_t, enum iflib_restart_event); +static void iavf_mbx_lost(struct iavf_sc *); +static void iavf_mbx_retry_detach(struct iavf_sc *); +static void iavf_mbx_retry_failed(if_ctx_t); +static void iavf_mbx_retry_prepare(struct iavf_sc *); +static void iavf_mbx_retry_stop(struct iavf_sc *); +static void iavf_mbx_retry_succeeded(struct iavf_sc *); +static int iavf_reestablish_vc(struct iavf_sc *); +static void iavf_replay_filters(struct iavf_sc *); +static int iavf_wait_asq(struct iavf_sc *, u32); + static int iavf_allocate_pci_resources(struct iavf_sc *); static void iavf_free_pci_resources(struct iavf_sc *); static void iavf_setup_interface(struct iavf_sc *); @@ -488,6 +511,10 @@ iavf_if_attach_post(if_ctx_t ctx) iavf_add_device_sysctls(sc); atomic_store_rel_32(&sc->queues_enabled, 0); + atomic_store_rel_32(&sc->mbx_ready, 1); + atomic_store_rel_32(&sc->vc_reinit_required, 0); + callout_init(&sc->mbx_retry, 1); + sc->mbx_retry_initialized = true; iavf_set_state(&sc->state, IAVF_STATE_INITIALIZED); /* We want AQ enabled early for init */ @@ -521,6 +548,7 @@ iavf_if_detach(if_ctx_t ctx) INIT_DBG_DEV(dev, "begin"); + iavf_mbx_retry_detach(sc); iavf_clear_state(&sc->state, IAVF_STATE_INITIALIZED); /* Drain admin queue taskqueue */ @@ -556,9 +584,9 @@ iavf_if_detach(if_ctx_t ctx) * @returns zero or an error code on failure */ static int -iavf_if_shutdown(if_ctx_t ctx __unused) +iavf_if_shutdown(if_ctx_t ctx) { - return (0); + return (iavf_if_suspend(ctx)); } /** @@ -570,8 +598,11 @@ iavf_if_shutdown(if_ctx_t ctx __unused) * @returns zero or an error code on failure */ static int -iavf_if_suspend(if_ctx_t ctx __unused) +iavf_if_suspend(if_ctx_t ctx) { + struct iavf_sc *sc = iavf_sc_from_ctx(ctx); + + iavf_mbx_retry_stop(sc); return (0); } @@ -645,7 +676,7 @@ iavf_send_vc_msg_sleep(struct iavf_sc *sc, u32 op) error = iavf_vc_sleep_wait(sc, op); IAVF_VC_LOCK_ASSERT(sc); - if (error == EWOULDBLOCK) + if (error == EWOULDBLOCK && iavf_mbx_log_allowed(sc)) device_printf(sc->dev, "%b timed out\n", op, IAVF_FLAGS); } release_lock: @@ -703,6 +734,255 @@ iavf_init_queues(struct iavf_vsi *vsi) } } +/* + * A VF can outlive a PF reset or temporary loss of virtchnl service. Keep + * repeated mailbox discovery out of ordinary status paths and retry complete + * initialization only while the interface remains administratively up. + */ +static void +iavf_mbx_retry_callout(void *arg) +{ + struct iavf_sc *sc; + if_t ifp; + + sc = arg; + if (atomic_readandclear_32(&sc->mbx_retry_pending) == 0 || + atomic_load_acq_32(&sc->mbx_ready) != 0 || + iflib_in_detach(sc->vsi.ctx)) + return; + ifp = iflib_get_ifp(sc->vsi.ctx); + if ((if_getflags(ifp) & IFF_UP) == 0) + return; + + iflib_request_reset_if_up(sc->vsi.ctx); + iflib_admin_intr_deferred(sc->vsi.ctx); +} + +bool +iavf_mbx_log_allowed(struct iavf_sc *sc) +{ + + /* Report each backoff stage, then limit the steady eight-second retry. */ + if (sc->mbx_retry_stage != nitems(iavf_mbx_retry_delay) - 1) + return (true); + return (ratecheck(&sc->mbx_last_log, &iavf_mbx_log_interval) != 0); +} + +static void +iavf_mbx_retry_detach(struct iavf_sc *sc) +{ + + if (!sc->mbx_retry_initialized) + return; + atomic_readandclear_32(&sc->mbx_retry_pending); + callout_drain(&sc->mbx_retry); + sc->mbx_retry_initialized = false; +} + +static void +iavf_mbx_retry_prepare(struct iavf_sc *sc) +{ + + if (!sc->mbx_retry_initialized) + return; + atomic_readandclear_32(&sc->mbx_retry_pending); + callout_drain(&sc->mbx_retry); +} + +static void +iavf_mbx_retry_stop(struct iavf_sc *sc) +{ + if_t ifp; + + if (!sc->mbx_retry_initialized) + return; + atomic_readandclear_32(&sc->mbx_retry_pending); + callout_drain(&sc->mbx_retry); + ifp = iflib_get_ifp(sc->vsi.ctx); + if ((if_getflags(ifp) & IFF_UP) == 0) + sc->mbx_retry_stage = 0; +} + +static void +iavf_mbx_retry_failed(if_ctx_t ctx) +{ + struct iavf_sc *sc; + struct iavf_vsi *vsi; + if_t ifp; + sbintime_t delay; + u_int stage; + + sc = iavf_sc_from_ctx(ctx); + vsi = &sc->vsi; + atomic_store_rel_32(&sc->mbx_ready, 0); + iavf_clear_state(&sc->state, IAVF_STATE_RUNNING); + sc->link_up = false; + if (vsi->link_active) { + vsi->link_active = false; + iflib_link_state_change(ctx, LINK_STATE_DOWN, 0); + } + iflib_init_failed(ctx); + + ifp = iflib_get_ifp(ctx); + if (!sc->mbx_retry_initialized || + (if_getflags(ifp) & IFF_UP) == 0) + return; + stage = sc->mbx_retry_stage; + if (stage >= nitems(iavf_mbx_retry_delay)) + stage = nitems(iavf_mbx_retry_delay) - 1; + delay = iavf_mbx_retry_delay[stage]; + if (sc->mbx_retry_stage + 1 < nitems(iavf_mbx_retry_delay)) + sc->mbx_retry_stage++; + atomic_set_32(&sc->mbx_retry_pending, 1); + callout_reset_sbt(&sc->mbx_retry, delay, 0, + iavf_mbx_retry_callout, sc, C_PREL(1)); +} + +static void +iavf_mbx_retry_succeeded(struct iavf_sc *sc) +{ + bool recovered; + + recovered = sc->mbx_retry_stage != 0; + atomic_store_rel_32(&sc->vc_reinit_required, 0); + atomic_store_rel_32(&sc->mbx_ready, 1); + atomic_readandclear_32(&sc->mbx_retry_pending); + if (sc->mbx_retry_initialized) + callout_stop(&sc->mbx_retry); + sc->mbx_retry_stage = 0; + sc->mbx_last_log.tv_sec = 0; + sc->mbx_last_log.tv_usec = 0; + iavf_clear_state(&sc->state, IAVF_STATE_RESET_REQUIRED); + iavf_clear_state(&sc->state, IAVF_STATE_RESET_PENDING); + if (recovered) + device_printf(sc->dev, "PF mailbox communication restored\n"); +} + +static void +iavf_mbx_lost(struct iavf_sc *sc) +{ + struct iavf_vsi *vsi; + + atomic_store_rel_32(&sc->vc_reinit_required, 1); + if (atomic_readandclear_32(&sc->mbx_ready) == 0) + return; + vsi = &sc->vsi; + iavf_clear_state(&sc->state, IAVF_STATE_RUNNING); + sc->link_up = false; + if (vsi->link_active) { + vsi->link_active = false; + iflib_link_state_change(vsi->ctx, LINK_STATE_DOWN, 0); + } + iflib_request_reset_if_up(vsi->ctx); + iflib_admin_intr_deferred(vsi->ctx); +} + +static int +iavf_wait_asq(struct iavf_sc *sc, u32 max_retries) +{ + struct iavf_hw *hw; + + hw = &sc->hw; + for (u32 retry = 0; retry < max_retries; retry++) { + if (iavf_asq_done(hw)) + return (0); + iavf_msec_pause(10); + } + return (ETIMEDOUT); +} + +/* + * A VFLR discards the Admin Queue and lets the PF replace the VF's VSI. + * Re-establish VERSION and GET_VF_RESOURCES before using any cached VSI ID. + * Runtime attempts are deliberately shorter than attach-time discovery; the + * retry callout supplies the longer backoff when the PF remains unavailable. + */ +static int +iavf_reestablish_vc(struct iavf_sc *sc) +{ + struct iavf_hw *hw; + struct iavf_vsi *vsi; + enum iavf_status status; + int error; + + hw = &sc->hw; + vsi = &sc->vsi; + iavf_disable_adminq_irq(hw); + taskqueue_drain(sc->vc_tq, &sc->vc_task); + /* A task already running when interrupts were masked can re-enable it. */ + iavf_disable_adminq_irq(hw); + pci_enable_busmaster(sc->dev); + + status = iavf_shutdown_adminq(hw); + if (status != IAVF_SUCCESS) + return (EIO); + status = iavf_init_adminq(hw); + if (status != IAVF_SUCCESS) + return (EIO); + + error = iavf_send_api_ver(sc); + if (error != 0) + goto fail; + error = iavf_wait_asq(sc, IAVF_MBX_RECOVERY_ASQ_RETRIES); + if (error != 0) + goto fail; + error = iavf_verify_api_ver_retries(sc, + IAVF_MBX_RECOVERY_VERSION_RETRIES); + if (error != 0) + goto fail; + + error = iavf_send_vf_config_msg(sc); + if (error != 0) + goto fail; + error = iavf_wait_asq(sc, IAVF_MBX_RECOVERY_ASQ_RETRIES); + if (error != 0) + goto fail; + error = iavf_get_vf_config_retries(sc, + IAVF_MBX_RECOVERY_CONFIG_RETRIES); + if (error != 0) + goto fail; + error = iavf_get_vsi_res_from_vf_res(sc); + if (error != 0) + goto fail; + + if (vsi->num_tx_queues > sc->vsi_res->num_queue_pairs || + vsi->num_rx_queues > sc->vsi_res->num_queue_pairs || + vsi->num_rx_queues + 1 > sc->vf_res->max_vectors) { + if (iavf_mbx_log_allowed(sc)) + device_printf(sc->dev, + "PF now provides %u queue pairs and %u vectors; " + "the VF has %u TX and %u RX queues\n", + sc->vsi_res->num_queue_pairs, + sc->vf_res->max_vectors, vsi->num_tx_queues, + vsi->num_rx_queues); + error = ENOSPC; + goto fail; + } + + iavf_enable_adminq_irq(hw); + return (0); + +fail: + iavf_disable_adminq_irq(hw); + return (error); +} + +static void +iavf_replay_filters(struct iavf_sc *sc) +{ + struct iavf_mac_filter *mac; + struct iavf_vlan_filter *vlan; + + SLIST_FOREACH(mac, sc->mac_filters, next) { + if ((mac->flags & IAVF_FILTER_DEL) == 0) + mac->flags |= IAVF_FILTER_ADD | IAVF_FILTER_USED; + } + SLIST_FOREACH(vlan, sc->vlan_filters, next) { + if ((vlan->flags & IAVF_FILTER_DEL) == 0) + vlan->flags = IAVF_FILTER_ADD; + } +} + /** * iavf_if_init - Initialize device for operation * @ctx: the iflib context pointer @@ -710,8 +990,9 @@ iavf_init_queues(struct iavf_vsi *vsi) * Initializes a device for operation. Called by iflib in response to an * interface up event from the stack. * - * @remark this function does not return a value and thus cannot indicate - * failure to initialize. + * Recoverable failures are reported to iflib with iflib_init_failed(), and a + * bounded callout retries initialization while the interface remains + * administratively up. */ static void iavf_if_init(if_ctx_t ctx) @@ -721,43 +1002,42 @@ iavf_if_init(if_ctx_t ctx) struct iavf_hw *hw = &sc->hw; if_t ifp = iflib_get_ifp(ctx); u8 tmpaddr[ETHER_ADDR_LEN]; - enum iavf_status status; device_t dev = sc->dev; + bool replay_filters; int error = 0; INIT_DBG_IF(ifp, "begin"); sx_assert(iflib_ctx_lock_get(ctx), SA_XLOCKED); + iavf_mbx_retry_prepare(sc); + replay_filters = atomic_load_acq_32(&sc->vc_reinit_required) != 0; - error = iavf_reset_complete(hw); - if (error) { - device_printf(sc->dev, "%s: VF reset failed\n", - __func__); + if (!iavf_reset_is_complete(hw)) { + atomic_store_rel_32(&sc->vc_reinit_required, 1); + if (iavf_mbx_log_allowed(sc)) + device_printf(dev, + "PF mailbox is unavailable; initialization deferred\n"); + iavf_mbx_retry_failed(ctx); + return; } - if (!iavf_check_asq_alive(hw)) { - iavf_dbg_info(sc, "ASQ is not alive, re-initializing AQ\n"); - pci_enable_busmaster(dev); - - status = iavf_shutdown_adminq(hw); - if (status != IAVF_SUCCESS) { - device_printf(dev, - "%s: iavf_shutdown_adminq failed: %s\n", - __func__, iavf_stat_str(hw, status)); - return; - } - - status = iavf_init_adminq(hw); - if (status != IAVF_SUCCESS) { - device_printf(dev, - "%s: iavf_init_adminq failed: %s\n", - __func__, iavf_stat_str(hw, status)); - return; + atomic_store_rel_32(&sc->vc_reinit_required, 1); + replay_filters = true; + } + if (replay_filters) { + error = iavf_reestablish_vc(sc); + if (error != 0) { + if (iavf_mbx_log_allowed(sc)) + device_printf(dev, + "PF mailbox rediscovery failed: %d\n", error); + goto fail; } } /* Make sure queues are disabled */ - iavf_disable_queues_with_retries(sc); + error = iavf_disable_queues_with_retries(sc); + if (error != 0) + goto fail; bcopy(if_getlladdr(ifp), tmpaddr, ETHER_ADDR_LEN); if (!cmp_etheraddr(hw->mac.addr, tmpaddr) && @@ -770,8 +1050,12 @@ iavf_if_init(if_ctx_t ctx) } error = iavf_add_mac_filter(sc, hw->mac.addr, 0); - if (!error || error == EEXIST) + if (replay_filters) + iavf_replay_filters(sc); + if (!error || error == EEXIST || replay_filters) iavf_send_vc_msg(sc, IAVF_FLAG_AQ_ADD_MAC_FILTER); + if (replay_filters) + iavf_send_vc_msg(sc, IAVF_FLAG_AQ_ADD_VLAN_FILTER); iflib_set_mac(ctx, hw->mac.addr); /* Prepare the queues for operation */ @@ -798,9 +1082,19 @@ iavf_if_init(if_ctx_t ctx) iavf_config_promisc(sc, if_getflags(ifp)); /* Enable queues */ - iavf_send_vc_msg_sleep(sc, IAVF_FLAG_AQ_ENABLE_QUEUES); + atomic_store_rel_32(&sc->queues_enabled, 0); + error = iavf_send_vc_msg_sleep(sc, IAVF_FLAG_AQ_ENABLE_QUEUES); + if (error != 0 || + atomic_load_acq_32(&sc->queues_enabled) == 0) + goto fail; + iavf_mbx_retry_succeeded(sc); iavf_set_state(&sc->state, IAVF_STATE_RUNNING); + return; + +fail: + atomic_store_rel_32(&sc->vc_reinit_required, 1); + iavf_mbx_retry_failed(ctx); } /** @@ -1253,8 +1547,19 @@ iavf_if_update_admin_status(if_ctx_t ctx) { struct iavf_sc *sc = iavf_sc_from_ctx(ctx); struct iavf_hw *hw = &sc->hw; + struct iavf_vsi *vsi = &sc->vsi; + if_t ifp = iflib_get_ifp(ctx); u16 pending = 0; + if ((if_getdrvflags(ifp) & IFF_DRV_RUNNING) == 0 || + atomic_load_acq_32(&sc->mbx_ready) == 0) { + if (vsi->link_active) { + vsi->link_active = false; + iflib_link_state_change(ctx, LINK_STATE_DOWN, 0); + } + return; + } + iavf_process_adminq(sc, &pending); iavf_update_link_status(sc); @@ -1378,12 +1683,16 @@ iavf_if_timer(if_ctx_t ctx, uint16_t qid) if (qid != 0) return; - /* Check for when PF triggers a VF reset */ + /* Check for a PF-triggered VF reset or a dead admin send queue. */ val = rd32(hw, IAVF_VFGEN_RSTAT) & IAVF_VFGEN_RSTAT_VFR_STATE_MASK; - if (val != VIRTCHNL_VFR_VFACTIVE - && val != VIRTCHNL_VFR_COMPLETED) { - iavf_dbg_info(sc, "reset in progress! (%d)\n", val); + if (iavf_test_state(&sc->state, IAVF_STATE_RESET_PENDING) || + !iavf_check_asq_alive(hw) || + (val != VIRTCHNL_VFR_VFACTIVE && + val != VIRTCHNL_VFR_COMPLETED)) { + iavf_dbg_info(sc, "PF mailbox unavailable (reset state %d)\n", + val); + iavf_mbx_lost(sc); return; } @@ -1784,11 +2093,29 @@ iavf_update_link_status(struct iavf_sc *sc) static void iavf_stop(struct iavf_sc *sc) { + struct iavf_vsi *vsi; + bool mailbox_ready; + + vsi = &sc->vsi; + iavf_mbx_retry_stop(sc); iavf_clear_state(&sc->state, IAVF_STATE_RUNNING); - iavf_disable_intr(&sc->vsi); + iavf_disable_intr(vsi); - iavf_disable_queues_with_retries(sc); + mailbox_ready = atomic_load_acq_32(&sc->mbx_ready) != 0; + if (mailbox_ready && iavf_reset_is_complete(&sc->hw) && + iavf_disable_queues_with_retries(sc) != 0) + mailbox_ready = false; + atomic_store_rel_32(&sc->mbx_ready, 0); + if (!mailbox_ready) { + atomic_store_rel_32(&sc->vc_reinit_required, 1); + iavf_dbg_vc(sc, "PF mailbox unavailable while stopping\n"); + } + sc->link_up = false; + if (vsi->link_active) { + vsi->link_active = false; + iflib_link_state_change(vsi->ctx, LINK_STATE_DOWN, 0); + } } /**