From nobody Mon Nov 10 15:51:26 2025 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4d4vKV6P7Hz6GtFw; Mon, 10 Nov 2025 15:51:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R12" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4d4vKV2q62z44CN; Mon, 10 Nov 2025 15:51:26 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1762789886; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1PNGzexlcp6IAmOWoTETGB0BTBHqBZptWOBgr/RTRRg=; b=LwBbVqmvcdlh9HMLZzLe3aHEpGslmct/kA4WCmFRn4mwMcy5zaD7VHW1PmLhpXj2W+da0G 2BptfWavi1MgK9gT6HAxaIndhhwwO9xq6N2Ltlq44GvOX51OcJjRD1Vvdx+SaqpDgZwWLr EZluB71M0BHWDzXIcjOgq7WAcXa2PMHzm/BsgnUT2bxwdGqDfBtz0E3UjC2Km/wpqZ1pBY PvcXPpzVmfzRm8+b2koEdhCoOG7mbBW3p3TdedduD3bgjr9XcfBWCwDOo6Hfg/ZVpf8SpV oEMIJARsYAf9OzkfCEOG8xk70dhZBBVxH2D4NfDdppHqG+vlXDW6kRnN3yXN8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1762789886; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1PNGzexlcp6IAmOWoTETGB0BTBHqBZptWOBgr/RTRRg=; b=pyIdmTjIZr72HroAsSkpx6NbWpYPHO8b8sJPwcZFe7c4V5d/t/jW0QXF/z0h9kEexZo5CQ hGwKpY9VDvzER94hWgw4kKqcuHChsUXo85blU5Iv121CsDvxCBulkELx6XldRkkWxUGuVG MfCM0PwO3x4c4VRQqrUQshlPhVDtAwmBL7SnUs1/+RIMm3KosNDw0WCFkIeoDCNse7ShM9 zFlbvQBScZO5Feld4TvU3bMxv1gWMrVCRQm86O8Ylc3SNpT19+UJqT7Ie4Xv9c/PHfO104 lR5Rizh3iS5vWHib3Vw3KmcowKZfmO1bfNMBeCiGovsl5KTAC+jWZlriBGbXPw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1762789886; a=rsa-sha256; cv=none; b=WhDzv+4OWm319VgSCP2jGReuo1xRm48DbwhIKc2UyPX7fFueF6siRiJfeasGwhKYQ0JAkJ 4XtBHl6//7WWh1u1mTHTrGJdfMwbdn2yThY+n67+dF6PaxvpoN87R3p1bdmzCystpRSL2v 898EG8u3YaqwmhH6EW4sFg50oyC1MT5SL63dE0i4OEciL+EvAMom3uJj8FNDj2W3gOxTjv tuneaFumgfk7h9KNdPSotL3DnfAdFcOF8U6cvYhaVqVnVTHTR/rTAWduw2HWRA5X3veFWP 5AYB8SG4zH5QvN1jn9XkkhUcHxRzage814aQjTh0XvCQaRgtK2RJSoT2CUa5pw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4d4vKV1phwz8nN; Mon, 10 Nov 2025 15:51:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 5AAFpQ1e048119; Mon, 10 Nov 2025 15:51:26 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 5AAFpQq9048116; Mon, 10 Nov 2025 15:51:26 GMT (envelope-from git) Date: Mon, 10 Nov 2025 15:51:26 GMT Message-Id: <202511101551.5AAFpQq9048116@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: adaa1470777b - main - cxgbe: Extend support for NVMe capabilities List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: adaa1470777b7b392960661e317577bf23fe029a Auto-Submitted: auto-generated The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=adaa1470777b7b392960661e317577bf23fe029a commit adaa1470777b7b392960661e317577bf23fe029a Author: John Baldwin AuthorDate: 2025-11-10 15:50:47 +0000 Commit: John Baldwin CommitDate: 2025-11-10 15:50:47 +0000 cxgbe: Extend support for NVMe capabilities - Default to enabling all NVMe capabilities. Currently, NVMe/TCP PDU offload is supported by T7 adapters. - Fetch relevant parameters from the firmware if NVMe support is enabled. A few parameters are shared with RDMA support, so the RDMA parameters section is now split into handling for parameters used by both vs parameters used only by RDMA. Sponsored by: Chelso Communications --- sys/dev/cxgbe/t4_main.c | 83 +++++++++++++++++++++++++++---------------------- 1 file changed, 46 insertions(+), 37 deletions(-) diff --git a/sys/dev/cxgbe/t4_main.c b/sys/dev/cxgbe/t4_main.c index 22d2f504c257..5bc503e48bf2 100644 --- a/sys/dev/cxgbe/t4_main.c +++ b/sys/dev/cxgbe/t4_main.c @@ -611,7 +611,7 @@ static int t4_switchcaps_allowed = FW_CAPS_CONFIG_SWITCH_INGRESS | SYSCTL_INT(_hw_cxgbe, OID_AUTO, switchcaps_allowed, CTLFLAG_RDTUN, &t4_switchcaps_allowed, 0, "Default switch capabilities"); -static int t4_nvmecaps_allowed = 0; +static int t4_nvmecaps_allowed = -1; SYSCTL_INT(_hw_cxgbe, OID_AUTO, nvmecaps_allowed, CTLFLAG_RDTUN, &t4_nvmecaps_allowed, 0, "Default NVMe capabilities"); @@ -5408,6 +5408,7 @@ apply_cfg_and_initialize(struct adapter *sc, char *cfg_file, caps.toecaps = 0; caps.rdmacaps = 0; caps.iscsicaps = 0; + caps.nvmecaps = 0; } caps.op_to_write = htobe32(V_FW_CMD_OP(FW_CAPS_CONFIG_CMD) | @@ -5881,61 +5882,63 @@ get_params__post_init(struct adapter *sc) * that will never be used. */ sc->iscsicaps = 0; + sc->nvmecaps = 0; sc->rdmacaps = 0; } - if (sc->rdmacaps) { + if (sc->nvmecaps || sc->rdmacaps) { param[0] = FW_PARAM_PFVF(STAG_START); param[1] = FW_PARAM_PFVF(STAG_END); - param[2] = FW_PARAM_PFVF(RQ_START); - param[3] = FW_PARAM_PFVF(RQ_END); - param[4] = FW_PARAM_PFVF(PBL_START); - param[5] = FW_PARAM_PFVF(PBL_END); - rc = -t4_query_params(sc, sc->mbox, sc->pf, 0, 6, param, val); + param[2] = FW_PARAM_PFVF(PBL_START); + param[3] = FW_PARAM_PFVF(PBL_END); + rc = -t4_query_params(sc, sc->mbox, sc->pf, 0, 4, param, val); if (rc != 0) { device_printf(sc->dev, - "failed to query RDMA parameters(1): %d.\n", rc); + "failed to query NVMe/RDMA parameters: %d.\n", rc); return (rc); } sc->vres.stag.start = val[0]; sc->vres.stag.size = val[1] - val[0] + 1; - sc->vres.rq.start = val[2]; - sc->vres.rq.size = val[3] - val[2] + 1; - sc->vres.pbl.start = val[4]; - sc->vres.pbl.size = val[5] - val[4] + 1; - - param[0] = FW_PARAM_PFVF(SQRQ_START); - param[1] = FW_PARAM_PFVF(SQRQ_END); - param[2] = FW_PARAM_PFVF(CQ_START); - param[3] = FW_PARAM_PFVF(CQ_END); - param[4] = FW_PARAM_PFVF(OCQ_START); - param[5] = FW_PARAM_PFVF(OCQ_END); + sc->vres.pbl.start = val[2]; + sc->vres.pbl.size = val[3] - val[2] + 1; + } + if (sc->rdmacaps) { + param[0] = FW_PARAM_PFVF(RQ_START); + param[1] = FW_PARAM_PFVF(RQ_END); + param[2] = FW_PARAM_PFVF(SQRQ_START); + param[3] = FW_PARAM_PFVF(SQRQ_END); + param[4] = FW_PARAM_PFVF(CQ_START); + param[5] = FW_PARAM_PFVF(CQ_END); rc = -t4_query_params(sc, sc->mbox, sc->pf, 0, 6, param, val); if (rc != 0) { device_printf(sc->dev, - "failed to query RDMA parameters(2): %d.\n", rc); + "failed to query RDMA parameters(1): %d.\n", rc); return (rc); } - sc->vres.qp.start = val[0]; - sc->vres.qp.size = val[1] - val[0] + 1; - sc->vres.cq.start = val[2]; - sc->vres.cq.size = val[3] - val[2] + 1; - sc->vres.ocq.start = val[4]; - sc->vres.ocq.size = val[5] - val[4] + 1; - - param[0] = FW_PARAM_PFVF(SRQ_START); - param[1] = FW_PARAM_PFVF(SRQ_END); - param[2] = FW_PARAM_DEV(MAXORDIRD_QP); - param[3] = FW_PARAM_DEV(MAXIRD_ADAPTER); - rc = -t4_query_params(sc, sc->mbox, sc->pf, 0, 4, param, val); + sc->vres.rq.start = val[0]; + sc->vres.rq.size = val[1] - val[0] + 1; + sc->vres.qp.start = val[2]; + sc->vres.qp.size = val[3] - val[2] + 1; + sc->vres.cq.start = val[4]; + sc->vres.cq.size = val[5] - val[4] + 1; + + param[0] = FW_PARAM_PFVF(OCQ_START); + param[1] = FW_PARAM_PFVF(OCQ_END); + param[2] = FW_PARAM_PFVF(SRQ_START); + param[3] = FW_PARAM_PFVF(SRQ_END); + param[4] = FW_PARAM_DEV(MAXORDIRD_QP); + param[5] = FW_PARAM_DEV(MAXIRD_ADAPTER); + rc = -t4_query_params(sc, sc->mbox, sc->pf, 0, 6, param, val); if (rc != 0) { device_printf(sc->dev, - "failed to query RDMA parameters(3): %d.\n", rc); + "failed to query RDMA parameters(2): %d.\n", rc); return (rc); } - sc->vres.srq.start = val[0]; - sc->vres.srq.size = val[1] - val[0] + 1; - sc->params.max_ordird_qp = val[2]; - sc->params.max_ird_adapter = val[3]; + sc->vres.ocq.start = val[0]; + sc->vres.ocq.size = val[1] - val[0] + 1; + sc->vres.srq.start = val[2]; + sc->vres.srq.size = val[3] - val[2] + 1; + sc->params.max_ordird_qp = val[4]; + sc->params.max_ird_adapter = val[5]; } if (sc->iscsicaps) { param[0] = FW_PARAM_PFVF(ISCSI_START); @@ -13694,6 +13697,9 @@ tweak_tunables(void) FW_CAPS_CONFIG_ISCSI_T10DIF; } + if (t4_nvmecaps_allowed == -1) + t4_nvmecaps_allowed = FW_CAPS_CONFIG_NVME_TCP; + if (t4_tmr_idx_ofld < 0 || t4_tmr_idx_ofld >= SGE_NTIMERS) t4_tmr_idx_ofld = TMR_IDX_OFLD; @@ -13705,6 +13711,9 @@ tweak_tunables(void) if (t4_iscsicaps_allowed == -1) t4_iscsicaps_allowed = 0; + + if (t4_nvmecaps_allowed == -1) + t4_nvmecaps_allowed = 0; #endif #ifdef DEV_NETMAP