From nobody Sat Jan 29 23:15:44 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 7ED02197AEF4; Sat, 29 Jan 2022 23:15:46 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JmVYX5BNbz3sNf; Sat, 29 Jan 2022 23:15:44 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1643498144; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=zkkk9YWmdj8/T0ttIAasCMAkohPcSHTgOk2jEfX9DLQ=; b=MZHNEtIX8F0Z80aKOokQ5yvdX4wnXawUhaPt9plc+iePXFVzhPIcmfNLNWfxpALz2sJvSM JGqn+p1WwNwDnDaSAFnw2Tg4sbE1wBHcG/4NmfF4Whl0E5t/2tn6MLgYm/cQ4oCN0GcRoc 9dyFZpmtmHZRSXDB/+Iij5Vm1W+o7lRzB+zOKxUyk3JKa7l1kk5lFGnbShKqjl/7nH+I1s kY9fphKodtPFm+GRlfcqla9Wu53+91uHeNwC4W2W/po3XvRYCrp3INpdrPzCThSzOTMV9w JpUPwkG9a9uHKsCHCAbLtrqh+Vb50tsa71h+IuLRXeMSuUAQOh2oUC1GXb4PzQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5B1BF1DF65; Sat, 29 Jan 2022 23:15:44 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 20TNFiJG008281; Sat, 29 Jan 2022 23:15:44 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 20TNFiEc008280; Sat, 29 Jan 2022 23:15:44 GMT (envelope-from git) Date: Sat, 29 Jan 2022 23:15:44 GMT Message-Id: <202201292315.20TNFiEc008280@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Chuck Tuffli Subject: git: ea9ee35583a8 - main - bhyve nvme: Add Temperature Threshold support List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: chuck X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: ea9ee35583a8faec1ec3d2f8df550baefc2b86b5 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1643498144; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=zkkk9YWmdj8/T0ttIAasCMAkohPcSHTgOk2jEfX9DLQ=; b=KTk46dJ2RwmRiOxlQmB/w4ZtUPR3V85esfV8wGUQNC6NNyp31dnSTrPgT0Te7CAz3BVkkj Ik40lJ7IM2tu0Sesjzz2YSm3aVmoeMbjteBePY+FAM7RUu61XwxwjBUsPyIlZaYauJWQ7J EdC24jXWRuFXbrpHVhELPkCILLv22hGnG/adF+yy4yzmWG8/gz0IRUpRSynuA7jtkKS8xa 2ZCsWzKt89RUp1kg95a4Q58ntVfGjR6sgueP4cw7B0MpI7YntthpaT5NCJt9sqF+WY++Nh y+nFWVUb5QjRB8sYQgVXlBIVC3pu4nubQt8OEA+DLj+qDChksm7ismNFxFWttw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1643498144; a=rsa-sha256; cv=none; b=OfpE/bnUfWFHbXJItndzAlIOWMD1XE9u/yCvUd1ISuboU3O5p/Im2AJU+Yuus7/qjINhPK dHV23gqGqZjDhp/t/ZWCFLb5nk2zg8GX1b0e9JNX/GmJQ6hCoIpFmBM7Fuh1vm5hdWvkVk cXYQA2TEvX5Y6gDgY7JmQ8ZnnURn8UAo51a3Eq9WiL/mK44y4QMKjphUCAVGYddEndEjmI Df4E7wf7UEzLhVbd0NQfOZMw1+kdFzDW0XpoCKu59Vbi/yI9LOqjXWRvQRpy3vqVDab/dG Y6FvJanLiAdpPx7GQPcSwQKGSqQ9c1+XVpFFN7KXaKEKQKu0QwZp/QeKATo42w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by chuck: URL: https://cgit.FreeBSD.org/src/commit/?id=ea9ee35583a8faec1ec3d2f8df550baefc2b86b5 commit ea9ee35583a8faec1ec3d2f8df550baefc2b86b5 Author: Chuck Tuffli AuthorDate: 2022-01-30 07:08:47 +0000 Commit: Chuck Tuffli CommitDate: 2022-01-30 07:08:47 +0000 bhyve nvme: Add Temperature Threshold support This adds the ability for a guest OS to send Set / Get Feature, Temperature Threshold commands. The implementation assumes a constant temperature and will generate an Asynchronous Event Notification if the specified threshold is above/below this value. Although the specification allows 9 temperature values, this implementation only implements the Composite Temperature. While in the neighborhood, move the clear of the CSTS register in the reset function after all other cleanup. This avoids a race with the guest thinking the reset is complete (i.e. CSTS.RDY = 0) before the NVMe emulation is actually complete with the reset. Fixes UNH IOL 16.0 Test 1.7, cases 1, 2, and 4. Tested by: jason@tubnor.net MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D33572 --- usr.sbin/bhyve/pci_nvme.c | 67 ++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 64 insertions(+), 3 deletions(-) diff --git a/usr.sbin/bhyve/pci_nvme.c b/usr.sbin/bhyve/pci_nvme.c index 9fe951a4796f..03338f6e0664 100644 --- a/usr.sbin/bhyve/pci_nvme.c +++ b/usr.sbin/bhyve/pci_nvme.c @@ -116,6 +116,9 @@ static int nvme_debug = 0; #define NVME_NO_STATUS 0xffff #define NVME_COMPLETION_VALID(c) ((c).status != NVME_NO_STATUS) +/* Reported temperature in Kelvin (i.e. room temperature) */ +#define NVME_TEMPERATURE 296 + /* helpers */ /* Convert a zero-based value into a one-based value */ @@ -392,6 +395,10 @@ static void nvme_feature_invalid_cb(struct pci_nvme_softc *, struct nvme_feature_obj *, struct nvme_command *, struct nvme_completion *); +static void nvme_feature_temperature(struct pci_nvme_softc *, + struct nvme_feature_obj *, + struct nvme_command *, + struct nvme_completion *); static void nvme_feature_num_queues(struct pci_nvme_softc *, struct nvme_feature_obj *, struct nvme_command *, @@ -523,6 +530,7 @@ pci_nvme_init_ctrldata(struct pci_nvme_softc *sc) /* Warning Composite Temperature Threshold */ cd->wctemp = 0x0157; + cd->cctemp = 0x0157; cd->sqes = (6 << NVME_CTRLR_DATA_SQES_MAX_SHIFT) | (6 << NVME_CTRLR_DATA_SQES_MIN_SHIFT); @@ -658,7 +666,7 @@ pci_nvme_init_logpages(struct pci_nvme_softc *sc) sc->write_dunits_remainder = 999; /* Set nominal Health values checked by implementations */ - sc->health_log.temperature = 310; + sc->health_log.temperature = NVME_TEMPERATURE; sc->health_log.available_spare = 100; sc->health_log.available_spare_threshold = 10; } @@ -672,7 +680,6 @@ pci_nvme_init_features(struct pci_nvme_softc *sc) switch (fid) { case NVME_FEAT_ARBITRATION: case NVME_FEAT_POWER_MANAGEMENT: - case NVME_FEAT_TEMPERATURE_THRESHOLD: case NVME_FEAT_INTERRUPT_COALESCING: //XXX case NVME_FEAT_WRITE_ATOMICITY: /* Mandatory but no special handling required */ @@ -680,6 +687,9 @@ pci_nvme_init_features(struct pci_nvme_softc *sc) //XXX hang - case NVME_FEAT_HOST_BEHAVIOR_SUPPORT: // this returns a data buffer break; + case NVME_FEAT_TEMPERATURE_THRESHOLD: + sc->feat[fid].set = nvme_feature_temperature; + break; case NVME_FEAT_ERROR_RECOVERY: sc->feat[fid].namespace_specific = true; break; @@ -1005,7 +1015,6 @@ pci_nvme_reset_locked(struct pci_nvme_softc *sc) sc->regs.vs = NVME_REV(1,4); /* NVMe v1.4 */ sc->regs.cc = 0; - sc->regs.csts = 0; assert(sc->submit_queues != NULL); @@ -1030,6 +1039,12 @@ pci_nvme_reset_locked(struct pci_nvme_softc *sc) pci_nvme_aer_destroy(sc); pci_nvme_aen_destroy(sc); + + /* + * Clear CSTS.RDY last to prevent the host from enabling Controller + * before cleanup completes + */ + sc->regs.csts = 0; } static void @@ -1630,7 +1645,53 @@ nvme_feature_iv_config(struct pci_nvme_softc *sc, pci_nvme_status_genc(&compl->status, NVME_SC_SUCCESS); } } +} + +#define NVME_TEMP_THRESH_OVER 0 +#define NVME_TEMP_THRESH_UNDER 1 +static void +nvme_feature_temperature(struct pci_nvme_softc *sc, + struct nvme_feature_obj *feat, + struct nvme_command *command, + struct nvme_completion *compl) +{ + uint16_t tmpth; /* Temperature Threshold */ + uint8_t tmpsel; /* Threshold Temperature Select */ + uint8_t thsel; /* Threshold Type Select */ + bool set_crit = false; + + tmpth = command->cdw11 & 0xffff; + tmpsel = (command->cdw11 >> 16) & 0xf; + thsel = (command->cdw11 >> 20) & 0x3; + + DPRINTF("%s: tmpth=%#x tmpsel=%#x thsel=%#x", __func__, tmpth, tmpsel, thsel); + + /* Check for unsupported values */ + if (((tmpsel != 0) && (tmpsel != 0xf)) || + (thsel > NVME_TEMP_THRESH_UNDER)) { + pci_nvme_status_genc(&compl->status, NVME_SC_INVALID_FIELD); + return; + } + + if (((thsel == NVME_TEMP_THRESH_OVER) && (NVME_TEMPERATURE >= tmpth)) || + ((thsel == NVME_TEMP_THRESH_UNDER) && (NVME_TEMPERATURE <= tmpth))) + set_crit = true; + + pthread_mutex_lock(&sc->mtx); + if (set_crit) + sc->health_log.critical_warning |= + NVME_CRIT_WARN_ST_TEMPERATURE; + else + sc->health_log.critical_warning &= + ~NVME_CRIT_WARN_ST_TEMPERATURE; + pthread_mutex_unlock(&sc->mtx); + + if (set_crit) + pci_nvme_aen_post(sc, PCI_NVME_AE_TYPE_SMART, + sc->health_log.critical_warning); + + DPRINTF("%s: set_crit=%c critical_warning=%#x status=%#x", __func__, set_crit ? 'T':'F', sc->health_log.critical_warning, compl->status); } static void