From nobody Fri Apr 15 20:46:51 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 040FC5D7574; Fri, 15 Apr 2022 20:46:53 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Kg7fh1Pvlz4tXd; Fri, 15 Apr 2022 20:46:52 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1650055612; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Tc9gBy02lYg1f4kx8ZbQgEolvD1w+odCYRkudwhzyRs=; b=vvmk7BhIEgkttqG28KdkEFwz5WbYOdw16HzKxGq6MwHoDQ6dGjnorodKP/zNIgZCQOCUfe cgWtr5b/kOK3SqlSXL0QOAtWbpXyvTiSwm/JRXi+Rx7bq9rYFubCawK/dMOORKxDxW4/Su i3sJ+hf8zaBVDVf4TXp1uzqSjEt+P/czO3zx2paoR3s5sOovAyYQEIGi0CjOgdQMHnNGeK /hCtGx5oefNRVGWupZcZU4Sqvf3rLw3YabbRnHrPtZIE+xMj24fwfqAj3mMbaEkZLPq7pj pn/M8Ggupex+AsfdfQs2pf3dm5dkOil4OaYLhA8Pk6t8wtk4d//lsKBlpU8nRw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E7A911C1EF; Fri, 15 Apr 2022 20:46:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 23FKkpvU079454; Fri, 15 Apr 2022 20:46:51 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 23FKkp30079453; Fri, 15 Apr 2022 20:46:51 GMT (envelope-from git) Date: Fri, 15 Apr 2022 20:46:51 GMT Message-Id: <202204152046.23FKkp30079453@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Warner Losh Subject: git: 0fd4cd405bbf - main - nvme: Use controller's page size instead of PAGE_SIZE to create qpair List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 0fd4cd405bbf6b9dcc3d4fc3ddf37c876ba97c11 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1650055612; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Tc9gBy02lYg1f4kx8ZbQgEolvD1w+odCYRkudwhzyRs=; b=v3u5QLU6f39V3UUmXYrCobouzIxviVvXQ0MlbC9eZ76KWVRyEYu0+rT7eilvJLOy/mPaRO TGwf5QwnOyi57yHAvQBpFzlygBe1DwOJZ1HwLjCc9Gk/f716rEWeK9/LY+7IQ/2CAO6A3s oVCMvI6G57n92M+rVOvMmPL0c3ylVp8r+gqrl493tij/aSUDww66ldYphcO/So8hypmsDN IncIagYcZroHKjE5JZ9dbS6uwVttspeOBQGEuAQpIBbSrvDyR0YXcccz3U5tj1MzcgL1ai n62gV6u+jqM9oth3lnfi7geVnb1MOlB43MG33rtMRbbCnb3ippH+ExpWAPNyVg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1650055612; a=rsa-sha256; cv=none; b=mZy2SUZkgC5yN3sHAebvYoce4nVlM4A1MiNVMipzJcQ9BGpWDJdaH+dz50rqTSfoKM3WW/ 4PIw6LSJe9kSCEDLByoBF4p/HQlglvM3gClY0c4bnAEAoJvlwTO1a1Gtm5aliWZMby5zt+ 20GAYRlrqTh2pdW/PUu89KK3pez6LxyDBjol7pmIbqIYdoSibs+yVxUPkjLlQvI04MNsql iYKT/UUTp5ewoyBbvJngxfOi55uLU2HgQ7DPbw6bmHzEC3GPa/WYgAQwhmGkm7PDDeMUAT HbnGPVJdnLuNz6pQ2zQSTqSDfPgiIkpfwIHGb/u/cQX4iGuc6/tEN1X5L3igTQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=0fd4cd405bbf6b9dcc3d4fc3ddf37c876ba97c11 commit 0fd4cd405bbf6b9dcc3d4fc3ddf37c876ba97c11 Author: Warner Losh AuthorDate: 2022-04-15 20:41:59 +0000 Commit: Warner Losh CommitDate: 2022-04-15 20:46:19 +0000 nvme: Use controller's page size instead of PAGE_SIZE to create qpair When constructing qpair, use the controller's notion of page size rather than the host's PAGE_SIZE. Currently, these are both 4k, but the arm 16k page size support requires decoupling. There's a "hidden" PAGE_SIZE in btoc, so we must change btoc(x) to howmany(x, ctrlr->page_size) to properly count the number of pages (in the drive's world view) are needed for various calculations. With these changes, we the nvme driver operates at production level load for both host 4k and host 16k page size. Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D34873 --- sys/dev/nvme/nvme_qpair.c | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/sys/dev/nvme/nvme_qpair.c b/sys/dev/nvme/nvme_qpair.c index 175975b2b3ac..3b20a7e209fa 100644 --- a/sys/dev/nvme/nvme_qpair.c +++ b/sys/dev/nvme/nvme_qpair.c @@ -702,9 +702,10 @@ nvme_qpair_construct(struct nvme_qpair *qpair, /* Note: NVMe PRP format is restricted to 4-byte alignment. */ err = bus_dma_tag_create(bus_get_dma_tag(ctrlr->dev), - 4, PAGE_SIZE, BUS_SPACE_MAXADDR, + 4, ctrlr->page_size, BUS_SPACE_MAXADDR, BUS_SPACE_MAXADDR, NULL, NULL, ctrlr->max_xfer_size, - btoc(ctrlr->max_xfer_size) + 1, PAGE_SIZE, 0, + howmany(ctrlr->max_xfer_size, ctrlr->page_size) + 1, + ctrlr->page_size, 0, NULL, NULL, &qpair->dma_tag_payload); if (err != 0) { nvme_printf(ctrlr, "payload tag create failed %d\n", err); @@ -716,20 +717,21 @@ nvme_qpair_construct(struct nvme_qpair *qpair, * cannot cross a page boundary. */ cmdsz = qpair->num_entries * sizeof(struct nvme_command); - cmdsz = roundup2(cmdsz, PAGE_SIZE); + cmdsz = roundup2(cmdsz, ctrlr->page_size); cplsz = qpair->num_entries * sizeof(struct nvme_completion); - cplsz = roundup2(cplsz, PAGE_SIZE); + cplsz = roundup2(cplsz, ctrlr->page_size); /* * For commands requiring more than 2 PRP entries, one PRP will be * embedded in the command (prp1), and the rest of the PRP entries * will be in a list pointed to by the command (prp2). */ - prpsz = sizeof(uint64_t) * btoc(ctrlr->max_xfer_size); + prpsz = sizeof(uint64_t) * + howmany(ctrlr->max_xfer_size, ctrlr->page_size); prpmemsz = qpair->num_trackers * prpsz; allocsz = cmdsz + cplsz + prpmemsz; err = bus_dma_tag_create(bus_get_dma_tag(ctrlr->dev), - PAGE_SIZE, 0, BUS_SPACE_MAXADDR, BUS_SPACE_MAXADDR, NULL, NULL, + ctrlr->page_size, 0, BUS_SPACE_MAXADDR, BUS_SPACE_MAXADDR, NULL, NULL, allocsz, 1, allocsz, 0, NULL, NULL, &qpair->dma_tag); if (err != 0) { nvme_printf(ctrlr, "tag create failed %d\n", err); @@ -791,13 +793,13 @@ nvme_qpair_construct(struct nvme_qpair *qpair, /* * Make sure that the PRP list for this tracker doesn't - * overflow to another page. + * overflow to another nvme page. */ if (trunc_page(list_phys) != trunc_page(list_phys + prpsz - 1)) { - list_phys = roundup2(list_phys, PAGE_SIZE); + list_phys = roundup2(list_phys, ctrlr->page_size); prp_list = - (uint8_t *)roundup2((uintptr_t)prp_list, PAGE_SIZE); + (uint8_t *)roundup2((uintptr_t)prp_list, ctrlr->page_size); } tr = malloc_domainset(sizeof(*tr), M_NVME, @@ -1101,10 +1103,9 @@ nvme_payload_map(void *arg, bus_dma_segment_t *seg, int nseg, int error) } /* - * Note that we specified PAGE_SIZE for alignment and max - * segment size when creating the bus dma tags. So here - * we can safely just transfer each segment to its - * associated PRP entry. + * Note that we specified ctrlr->page_size for alignment and max + * segment size when creating the bus dma tags. So here we can safely + * just transfer each segment to its associated PRP entry. */ tr->req->cmd.prp1 = htole64(seg[0].ds_addr);