From nobody Tue Feb 08 15:14:05 2022 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 03F0919C3C8A; Tue, 8 Feb 2022 15:14:10 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JtRPF0XqDz4dWZ; Tue, 8 Feb 2022 15:14:08 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1644333249; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=G+tOpfC8fjDRQUyl91iE3B5AwnN0tCNX1Vp3+TMyTc4=; b=LwzycmngekUCAfVFZ9jLiTFhK22PkAKigSqZ5V61NbL+VH19IIHla0e+nnZyGGVPLuKTI1 fLMCILOfhbKuvi5EtbIJx9rmum2U7dUyQcP581QOEuKCtPVfiq2qAm1gd3XyQG3IMxtyv4 jfKYmzvuAcy7tZQIVkJmeuJMrqCaBchTks6ehECOjUNWkXlIVd2W5nm4Dj3tAEOXIsa/+0 CQD9knvc0x4+ZEC2Z/KTtE0Q1ndiRtQ2A5HvgRAqT4LaZRIuQDTZaUAhpmRlDju7bPedr6 wSUSko+kbtOwia15Dg4mqi8qOLcdxb5AaMDzG/ZSLjse9xwjbbbPoEc63cN/Kw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 343571B9D9; Tue, 8 Feb 2022 15:14:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 218FE5av020812; Tue, 8 Feb 2022 15:14:05 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 218FE5QS020811; Tue, 8 Feb 2022 15:14:05 GMT (envelope-from git) Date: Tue, 8 Feb 2022 15:14:05 GMT Message-Id: <202202081514.218FE5QS020811@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Hans Petter Selasky Subject: git: e441376f455e - stable/13 - mlx5en: Add race protection for SQ remap List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: hselasky X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: e441376f455e7b68f07471d6283adf1e68f9e089 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1644333249; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=G+tOpfC8fjDRQUyl91iE3B5AwnN0tCNX1Vp3+TMyTc4=; b=qmr3JHojXitq16Xiv7JXkdAz5AjYBKaec03pL8JGu0w4/zDdLNgzvXg4sthSZgjYVOQAGy 2y9o+i44XVpMAmu/VaBKlxTHMXJVA96Vk4nInzpg2kWrjykUSJPOXvfov0vISqtDYIXh5G KEoAqNw9aR7AjDGeaYwGvomHGSkffDI4UWsiDFh1iYQO+ay+FbuBa4/WKOuSTsMEDYL9eH 8CPFjS/kWvQJ1DfINCytw+kph7yz60U6MXODwtR48LkMGWh3C+AKKJqkHcFyls6iI+ciyu Qu3oSJcdtDBSDODLsMYaO6XNXv/g2hcQMIiHKTMqBwNVLRSmNNKLZwOGWW0Wgg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1644333249; a=rsa-sha256; cv=none; b=XFcYxOdTZu0noCvM5Z8q1y2lONq4fiyiikpPXkebaeoPXj/HKiNno+ix9r7yBgwUjOuCsd L+tfTUSq6aVzXmBE1vvfOhvdQMc7ElIK21h52wsEU1S/kaqAOC5PKlft1pLFyi0kt4+TKA tcfb9s9zCp3CiplYDZQVmmb2ml9V/ppYd/w0D7x4QGpOfdIZ1OhF2OP2CvRwAJQYvXkvem aqN08AiA8IHD02XnhNfU9+WQ+XCLaZSqbyaGT1kl5jp27GL7vGtgiocvUitNKwcF7oZPdt L3ZhMGQPStlXSC4/CN7ayNRAhFTafzMQM4HP+Orlfk6c8802Ae46AsM9DraXvg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by hselasky: URL: https://cgit.FreeBSD.org/src/commit/?id=e441376f455e7b68f07471d6283adf1e68f9e089 commit e441376f455e7b68f07471d6283adf1e68f9e089 Author: Hans Petter Selasky AuthorDate: 2022-02-08 15:08:53 +0000 Commit: Hans Petter Selasky CommitDate: 2022-02-08 15:08:53 +0000 mlx5en: Add race protection for SQ remap Add a refcount for posted WQEs to avoid a race between post WQE and FW command flows. Sponsored by: NVIDIA Networking (cherry picked from commit a8e715d21b963251e449187c98292fff77dc7576) --- sys/dev/mlx5/mlx5_en/en_rl.h | 1 + sys/dev/mlx5/mlx5_en/mlx5_en_rl.c | 20 +++++++++++++++----- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/sys/dev/mlx5/mlx5_en/en_rl.h b/sys/dev/mlx5/mlx5_en/en_rl.h index f30e8ba8cc07..1aceef34929d 100644 --- a/sys/dev/mlx5/mlx5_en/en_rl.h +++ b/sys/dev/mlx5/mlx5_en/en_rl.h @@ -136,6 +136,7 @@ struct mlx5e_rl_channel { uint64_t new_rate; uint64_t init_rate; uint64_t last_rate; + uint32_t refcount; uint16_t last_burst; uint16_t state; }; diff --git a/sys/dev/mlx5/mlx5_en/mlx5_en_rl.c b/sys/dev/mlx5/mlx5_en/mlx5_en_rl.c index 8ab118c21249..26f6fd65c576 100644 --- a/sys/dev/mlx5/mlx5_en/mlx5_en_rl.c +++ b/sys/dev/mlx5/mlx5_en/mlx5_en_rl.c @@ -414,7 +414,8 @@ mlx5e_rl_find_best_rate_locked(struct mlx5e_rl_priv_data *rl, uint64_t user_rate } static int -mlx5e_rl_post_sq_remap_wqe(struct mlx5e_iq *iq, u32 scq_handle, u32 sq_handle) +mlx5e_rl_post_sq_remap_wqe(struct mlx5e_iq *iq, u32 scq_handle, u32 sq_handle, + struct mlx5e_rl_channel *sq_channel) { const u32 ds_cnt = DIV_ROUND_UP(sizeof(struct mlx5e_tx_qos_remap_wqe), MLX5_SEND_WQE_DS); @@ -444,6 +445,8 @@ mlx5e_rl_post_sq_remap_wqe(struct mlx5e_iq *iq, u32 scq_handle, u32 sq_handle) memcpy(iq->doorbell.d32, &wqe->ctrl, sizeof(iq->doorbell.d32)); iq->data[pi].num_wqebbs = DIV_ROUND_UP(ds_cnt, MLX5_SEND_WQEBB_NUM_DS); + iq->data[pi].p_refcount = &sq_channel->refcount; + atomic_add_int(iq->data[pi].p_refcount, 1); iq->pc += iq->data[pi].num_wqebbs; mlx5e_iq_notify_hw(iq); @@ -454,7 +457,8 @@ mlx5e_rl_post_sq_remap_wqe(struct mlx5e_iq *iq, u32 scq_handle, u32 sq_handle) } static int -mlx5e_rl_remap_sq(struct mlx5e_sq *sq, uint16_t index) +mlx5e_rl_remap_sq(struct mlx5e_sq *sq, uint16_t index, + struct mlx5e_rl_channel *sq_channel) { struct mlx5e_channel *iq_channel; u32 scq_handle; @@ -467,10 +471,12 @@ mlx5e_rl_remap_sq(struct mlx5e_sq *sq, uint16_t index) sq_handle = sq->queue_handle; scq_handle = mlx5_rl_get_scq_handle(sq->priv->mdev, index); - if (sq_handle == -1U || scq_handle == -1U) + if (sq_handle == MLX5_INVALID_QUEUE_HANDLE || + scq_handle == MLX5_INVALID_QUEUE_HANDLE) error = -1; else - error = mlx5e_rl_post_sq_remap_wqe(&iq_channel->iq, scq_handle, sq_handle); + error = mlx5e_rl_post_sq_remap_wqe(&iq_channel->iq, scq_handle, + sq_handle, sq_channel); return (error); } @@ -558,7 +564,11 @@ mlx5e_rlw_channel_set_rate_locked(struct mlx5e_rl_worker *rlw, /* set new rate, if SQ is running */ sq = channel->sq; if (sq != NULL && READ_ONCE(sq->running) != 0) { - if (!use_sq_remap || mlx5e_rl_remap_sq(sq, index)) { + if (!use_sq_remap || mlx5e_rl_remap_sq(sq, index, channel)) { + while (atomic_load_int(&channel->refcount) != 0 && + rlw->priv->mdev->state != MLX5_DEVICE_STATE_INTERNAL_ERROR && + pci_channel_offline(rlw->priv->mdev->pdev) == 0) + pause("W", 1); error = mlx5e_rl_modify_sq(sq, index); if (error != 0) atomic_add_64(&rlw->priv->rl.stats.tx_modify_rate_failure, 1ULL);