From nobody Mon Feb 21 21:57:58 2022 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9FC7C19EB3A1; Mon, 21 Feb 2022 21:57:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4K2blC2xLGz4gcL; Mon, 21 Feb 2022 21:57:59 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1645480679; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=aa3FEEXIBIKBaRfxa3nm/FQ1jpQ08utFdhcbi8rdFCc=; b=ow0CXlEruLzpzm+aLaAiu6lpouv0rdjL+BhfX+EcaOCApHIMfnADfKZvXLUt5tO6y/+u/A kcW484/bk1eFrSrAKhDcNGxPxkyMlFM1sAJyhYbA3pGp4qCy/3E+w/fwwKDo5aaItg/u6R 06jrCAHStukpmf6EEgs0MDdX2gYGtZXYs58pcCTkoJFXVVWIRRN4iWKuYDbrC4Srxgsjng vXz82w3YihdTKHtjL8WRwZs0sDZ44lh9taiOCGpJ0BBL/su75OBb2fr3Po4HOeTsry1t73 EPAFQ+q7Bsrvry6zl8ZGmq2DDHvLnWDcXcCk1J3G5L+VW4jqfg5qvkdAwN4K+w== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id ECA941849C; Mon, 21 Feb 2022 21:57:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 21LLvw6Y052160; Mon, 21 Feb 2022 21:57:58 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 21LLvw82052159; Mon, 21 Feb 2022 21:57:58 GMT (envelope-from git) Date: Mon, 21 Feb 2022 21:57:58 GMT Message-Id: <202202212157.21LLvw82052159@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Chuck Tuffli Subject: git: 0bb86659cbca - stable/13 - bhyve nvme: Add Temperature Threshold support List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: chuck X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 0bb86659cbcabcd8ec7d1a7eecefa6be2192c84d Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1645480679; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=aa3FEEXIBIKBaRfxa3nm/FQ1jpQ08utFdhcbi8rdFCc=; b=Hq3snnOjbVXF+Fqeys/kq7icEtxguqFOINsgTUoOISD6Q/5ybZ+8oP/Ncw+R7FD0YfxzQ9 EDc10ltOU34ENp7saFSP4X0+iq0KGkw/xjpXK4pNwSy+5nmHkikSxiurBAKpz9+946Xxzj f2pUz8naKF166KryyY4H5BSj6sIWHPoTnuYQYbUsq2Bv/DzYaI9Vrhg1XiodT/EhvyFGuT a5aPMa7MC31e8rwuhb+5n8XeDNAJ+T5Vx05lfgBPChHq/I8JvUa6jAnRr47s3VtGEx4MMN 2vQFUnj+3A3x5ChrZzslgC3d5ib6IAEiqrcmbycpGiWOc0vST2TBQcsiQaSOQQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1645480679; a=rsa-sha256; cv=none; b=ysVEeGuca1l/kFf4SZDpco97np+nV7VCZmmNwhE9aDcX1gtfK8oXC/UqxVfHyTRrurpUQL RLyZe6fEt4NGUmSJyIHC1QeSPHJ3UUj+PDSirwCOYdDbv5845sEOgXuWrALZ5jxPz7cFmk U7h7SUXkQepigRXviewB8aEVPRMuh3Zuxwp+4MLIaVEC1H+J4HI8s3CcAKfVPHpX2GA/5V oJScVNup2WTEsq0HwPG8phWc43CpBOOfEQP6uyv5GpNe32q1VAjF+E/Ip2xC7szjjLD+ZJ h5wdXaKjkd+tXIBC2+S9xvU00/q1ccP/ayqnwr0IBKmNE39DLv4ovjVg8l5dLg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by chuck: URL: https://cgit.FreeBSD.org/src/commit/?id=0bb86659cbcabcd8ec7d1a7eecefa6be2192c84d commit 0bb86659cbcabcd8ec7d1a7eecefa6be2192c84d Author: Chuck Tuffli AuthorDate: 2022-01-30 07:08:47 +0000 Commit: Chuck Tuffli CommitDate: 2022-02-22 03:26:45 +0000 bhyve nvme: Add Temperature Threshold support This adds the ability for a guest OS to send Set / Get Feature, Temperature Threshold commands. The implementation assumes a constant temperature and will generate an Asynchronous Event Notification if the specified threshold is above/below this value. Although the specification allows 9 temperature values, this implementation only implements the Composite Temperature. While in the neighborhood, move the clear of the CSTS register in the reset function after all other cleanup. This avoids a race with the guest thinking the reset is complete (i.e. CSTS.RDY = 0) before the NVMe emulation is actually complete with the reset. Fixes UNH IOL 16.0 Test 1.7, cases 1, 2, and 4. (cherry picked from commit ea9ee35583a8faec1ec3d2f8df550baefc2b86b5) --- usr.sbin/bhyve/pci_nvme.c | 67 ++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 64 insertions(+), 3 deletions(-) diff --git a/usr.sbin/bhyve/pci_nvme.c b/usr.sbin/bhyve/pci_nvme.c index 5719a5f6c8c2..94eb2137d1e6 100644 --- a/usr.sbin/bhyve/pci_nvme.c +++ b/usr.sbin/bhyve/pci_nvme.c @@ -116,6 +116,9 @@ static int nvme_debug = 0; #define NVME_NO_STATUS 0xffff #define NVME_COMPLETION_VALID(c) ((c).status != NVME_NO_STATUS) +/* Reported temperature in Kelvin (i.e. room temperature) */ +#define NVME_TEMPERATURE 296 + /* helpers */ /* Convert a zero-based value into a one-based value */ @@ -392,6 +395,10 @@ static void nvme_feature_invalid_cb(struct pci_nvme_softc *, struct nvme_feature_obj *, struct nvme_command *, struct nvme_completion *); +static void nvme_feature_temperature(struct pci_nvme_softc *, + struct nvme_feature_obj *, + struct nvme_command *, + struct nvme_completion *); static void nvme_feature_num_queues(struct pci_nvme_softc *, struct nvme_feature_obj *, struct nvme_command *, @@ -523,6 +530,7 @@ pci_nvme_init_ctrldata(struct pci_nvme_softc *sc) /* Warning Composite Temperature Threshold */ cd->wctemp = 0x0157; + cd->cctemp = 0x0157; cd->sqes = (6 << NVME_CTRLR_DATA_SQES_MAX_SHIFT) | (6 << NVME_CTRLR_DATA_SQES_MIN_SHIFT); @@ -658,7 +666,7 @@ pci_nvme_init_logpages(struct pci_nvme_softc *sc) sc->write_dunits_remainder = 999; /* Set nominal Health values checked by implementations */ - sc->health_log.temperature = 310; + sc->health_log.temperature = NVME_TEMPERATURE; sc->health_log.available_spare = 100; sc->health_log.available_spare_threshold = 10; } @@ -672,7 +680,6 @@ pci_nvme_init_features(struct pci_nvme_softc *sc) switch (fid) { case NVME_FEAT_ARBITRATION: case NVME_FEAT_POWER_MANAGEMENT: - case NVME_FEAT_TEMPERATURE_THRESHOLD: case NVME_FEAT_INTERRUPT_COALESCING: //XXX case NVME_FEAT_WRITE_ATOMICITY: /* Mandatory but no special handling required */ @@ -680,6 +687,9 @@ pci_nvme_init_features(struct pci_nvme_softc *sc) //XXX hang - case NVME_FEAT_HOST_BEHAVIOR_SUPPORT: // this returns a data buffer break; + case NVME_FEAT_TEMPERATURE_THRESHOLD: + sc->feat[fid].set = nvme_feature_temperature; + break; case NVME_FEAT_ERROR_RECOVERY: sc->feat[fid].namespace_specific = true; break; @@ -1005,7 +1015,6 @@ pci_nvme_reset_locked(struct pci_nvme_softc *sc) sc->regs.vs = NVME_REV(1,4); /* NVMe v1.4 */ sc->regs.cc = 0; - sc->regs.csts = 0; assert(sc->submit_queues != NULL); @@ -1030,6 +1039,12 @@ pci_nvme_reset_locked(struct pci_nvme_softc *sc) pci_nvme_aer_destroy(sc); pci_nvme_aen_destroy(sc); + + /* + * Clear CSTS.RDY last to prevent the host from enabling Controller + * before cleanup completes + */ + sc->regs.csts = 0; } static void @@ -1630,7 +1645,53 @@ nvme_feature_iv_config(struct pci_nvme_softc *sc, pci_nvme_status_genc(&compl->status, NVME_SC_SUCCESS); } } +} + +#define NVME_TEMP_THRESH_OVER 0 +#define NVME_TEMP_THRESH_UNDER 1 +static void +nvme_feature_temperature(struct pci_nvme_softc *sc, + struct nvme_feature_obj *feat, + struct nvme_command *command, + struct nvme_completion *compl) +{ + uint16_t tmpth; /* Temperature Threshold */ + uint8_t tmpsel; /* Threshold Temperature Select */ + uint8_t thsel; /* Threshold Type Select */ + bool set_crit = false; + + tmpth = command->cdw11 & 0xffff; + tmpsel = (command->cdw11 >> 16) & 0xf; + thsel = (command->cdw11 >> 20) & 0x3; + + DPRINTF("%s: tmpth=%#x tmpsel=%#x thsel=%#x", __func__, tmpth, tmpsel, thsel); + + /* Check for unsupported values */ + if (((tmpsel != 0) && (tmpsel != 0xf)) || + (thsel > NVME_TEMP_THRESH_UNDER)) { + pci_nvme_status_genc(&compl->status, NVME_SC_INVALID_FIELD); + return; + } + + if (((thsel == NVME_TEMP_THRESH_OVER) && (NVME_TEMPERATURE >= tmpth)) || + ((thsel == NVME_TEMP_THRESH_UNDER) && (NVME_TEMPERATURE <= tmpth))) + set_crit = true; + + pthread_mutex_lock(&sc->mtx); + if (set_crit) + sc->health_log.critical_warning |= + NVME_CRIT_WARN_ST_TEMPERATURE; + else + sc->health_log.critical_warning &= + ~NVME_CRIT_WARN_ST_TEMPERATURE; + pthread_mutex_unlock(&sc->mtx); + + if (set_crit) + pci_nvme_aen_post(sc, PCI_NVME_AE_TYPE_SMART, + sc->health_log.critical_warning); + + DPRINTF("%s: set_crit=%c critical_warning=%#x status=%#x", __func__, set_crit ? 'T':'F', sc->health_log.critical_warning, compl->status); } static void