From nobody Wed Apr 12 07:22:36 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PxDgc49qZz45RKx; Wed, 12 Apr 2023 07:22:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PxDgc3h7Yz4Hy5; Wed, 12 Apr 2023 07:22:36 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681284156; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=+pRby1TsacI7c87+G7AjCnfJesuwW2SVQK6BwrLtwSA=; b=TWzf6EwM5LWbdSWYWKCCBOPJiIHWpNC3xRMkVyPEmSiX0xuNwlkDbZ3aDHRgo+AVdSK7v1 BE3X9JIV+5E9FGwMgfI8riLAmP65hopbtjHQY+I2bITj+DkqzJY6zN5075tCNtmW0GMWNG bGBa6/r3oLB1iDTvFaWS4oEjIhAHyRcRlMfYdInNsIOEXjxea0G2F5XPXn51dkN2kEWYDZ TL/X3vVe56yx5Q9qoNwpezetfZ0cY/1/GTB5AR1CB6kJvCFFypd87nhKEXL8wsJ6ZqvInz JHs1/Q4YXRBnKD62dIAnHFfjofHno7j8gG5olHxGN/oQ9E+2jJBuaKp83yiPvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681284156; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=+pRby1TsacI7c87+G7AjCnfJesuwW2SVQK6BwrLtwSA=; b=i5W2lk73TFtHDO2fP5llo6nb2ZXxI+uT2XkIytYgb1WsILiHMhbHv2Vu3+DSdkKv8op8aI rod5IMTHyRqrGzVQDLp292PRnIDHJpC3OqnCQQOOer3JRj99YyGcAnDDFtYy7ABi5DjZez CEcvFq+/Oy7llH2yPjBx8Wk3n67tvgW+4aTPg97zbMq9wt8O28Xvi/ve+7fPpcJgEHpEdX 34xM1cgoLAt42BAbD9J9tnwT9+2YI5m6aTEIoo758Jyech3B9sNPh7xPlSjSnRTk8dvmqP Z2kGTDtZrIcvftDSaAp7RV8JfMCFC/Ci6+qKS4mfJDpJfnFufp1mVMv/5TH8lg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1681284156; a=rsa-sha256; cv=none; b=QDPpqab6fcZGzzM3XE8SKw4rHVa+0x69NkAXJiSpCU2IhMnDjLhSOm70+mDnLZZGMB5aHN 8/YkLIozEBR0Ik6wq/YxEsBZaszQueCi153/MyYBU3L4V0fAzC+wfwEdn498jw+Y5yy/YD +LCtiJCdXRdH7UGc1xjXsJfea9R5ap8tO1YCcyyGVlSbX2ouszCEoWDh5gJvRsWW0vMMQN oe8yGyBsN3611AmSZDwjb06W2wg+PGp/ccPsaEf07PLvmK62Ow/pw75OG+wvTLEk8d9Zb0 VSDFx09D0SXz7ZR7APAv5zvDpk+wXzoWHZzDCpFxjqLmM0h7kOqR6HAXnWmCTg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4PxDgc2kkMzrDW; Wed, 12 Apr 2023 07:22:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 33C7MaiT094652; Wed, 12 Apr 2023 07:22:36 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 33C7MadI094651; Wed, 12 Apr 2023 07:22:36 GMT (envelope-from git) Date: Wed, 12 Apr 2023 07:22:36 GMT Message-Id: <202304120722.33C7MadI094651@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: =?utf-8?Q?Corvin=20K=C3=B6hne?= Subject: git: 931bb7bf1c8f - main - bhyve: define array to protect passthru regs List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: corvink X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 931bb7bf1c8f90f3303b5291e4868c587f9364e7 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by corvink: URL: https://cgit.FreeBSD.org/src/commit/?id=931bb7bf1c8f90f3303b5291e4868c587f9364e7 commit 931bb7bf1c8f90f3303b5291e4868c587f9364e7 Author: Corvin Köhne AuthorDate: 2021-03-19 12:48:34 +0000 Commit: Corvin Köhne CommitDate: 2023-04-12 07:22:19 +0000 bhyve: define array to protect passthru regs GPU passthrough requires a special handling of some PCI config register. Therefore, we need a flexible approach for implementing it. Adding an array of handler meets this condition. Start by using the default handler for all accesses to the PCI config space. In upcoming commits, we can start to split the default handler into several handler for each register that requires emulation. Reviewed by: markj MFC after: 1 week Sponsored by: Beckhoff Automation GmbH & Co. KG Differential Revision: https://reviews.freebsd.org/D39291 --- usr.sbin/bhyve/pci_passthru.c | 57 +++++++++++++++++++++++++++++++++++++------ usr.sbin/bhyve/pci_passthru.h | 9 +++++++ 2 files changed, 58 insertions(+), 8 deletions(-) diff --git a/usr.sbin/bhyve/pci_passthru.c b/usr.sbin/bhyve/pci_passthru.c index d4382f6cc20e..0055de461270 100644 --- a/usr.sbin/bhyve/pci_passthru.c +++ b/usr.sbin/bhyve/pci_passthru.c @@ -93,6 +93,9 @@ struct passthru_softc { int capoff; } psc_msix; struct pcisel psc_sel; + + cfgread_handler psc_pcir_rhandler[PCI_REGMAX + 1]; + cfgwrite_handler psc_pcir_whandler[PCI_REGMAX + 1]; }; static int @@ -643,6 +646,23 @@ done: return (error); } +int +set_pcir_handler(struct passthru_softc *sc, int reg, int len, + cfgread_handler rhandler, cfgwrite_handler whandler) +{ + if (reg > PCI_REGMAX || reg + len > PCI_REGMAX + 1) + return (-1); + + for (int i = reg; i < reg + len; ++i) { + assert(sc->psc_pcir_rhandler[i] == NULL || rhandler == NULL); + assert(sc->psc_pcir_whandler[i] == NULL || whandler == NULL); + sc->psc_pcir_rhandler[i] = rhandler; + sc->psc_pcir_whandler[i] = whandler; + } + + return (0); +} + static int passthru_legacy_config(nvlist_t *nvl, const char *opts) { @@ -902,12 +922,9 @@ msixcap_access(struct passthru_softc *sc, int coff) } static int -passthru_cfgread(struct pci_devinst *pi, int coff, int bytes, uint32_t *rv) +passthru_cfgread_default(struct passthru_softc *sc, + struct pci_devinst *pi __unused, int coff, int bytes, uint32_t *rv) { - struct passthru_softc *sc; - - sc = pi->pi_arg; - /* * PCI BARs and MSI capability is emulated. */ @@ -946,14 +963,25 @@ passthru_cfgread(struct pci_devinst *pi, int coff, int bytes, uint32_t *rv) } static int -passthru_cfgwrite(struct pci_devinst *pi, int coff, int bytes, uint32_t val) +passthru_cfgread(struct pci_devinst *pi, int coff, int bytes, uint32_t *rv) { - int error, msix_table_entries, i; struct passthru_softc *sc; - uint16_t cmd_old; sc = pi->pi_arg; + if (sc->psc_pcir_rhandler[coff] != NULL) + return (sc->psc_pcir_rhandler[coff](sc, pi, coff, bytes, rv)); + + return (passthru_cfgread_default(sc, pi, coff, bytes, rv)); +} + +static int +passthru_cfgwrite_default(struct passthru_softc *sc, struct pci_devinst *pi, + int coff, int bytes, uint32_t val) +{ + int error, msix_table_entries, i; + uint16_t cmd_old; + /* * PCI BARs are emulated */ @@ -1026,6 +1054,19 @@ passthru_cfgwrite(struct pci_devinst *pi, int coff, int bytes, uint32_t val) return (0); } +static int +passthru_cfgwrite(struct pci_devinst *pi, int coff, int bytes, uint32_t val) +{ + struct passthru_softc *sc; + + sc = pi->pi_arg; + + if (sc->psc_pcir_whandler[coff] != NULL) + return (sc->psc_pcir_whandler[coff](sc, pi, coff, bytes, val)); + + return (passthru_cfgwrite_default(sc, pi, coff, bytes, val)); +} + static void passthru_write(struct pci_devinst *pi, int baridx, uint64_t offset, int size, uint64_t value) diff --git a/usr.sbin/bhyve/pci_passthru.h b/usr.sbin/bhyve/pci_passthru.h index bdc8dc3cd0e2..f6ce2d4b85ab 100644 --- a/usr.sbin/bhyve/pci_passthru.h +++ b/usr.sbin/bhyve/pci_passthru.h @@ -11,5 +11,14 @@ #include "pci_emul.h" +struct passthru_softc; + +typedef int (*cfgread_handler)(struct passthru_softc *sc, + struct pci_devinst *pi, int coff, int bytes, uint32_t *rv); +typedef int (*cfgwrite_handler)(struct passthru_softc *sc, + struct pci_devinst *pi, int coff, int bytes, uint32_t val); + uint32_t read_config(const struct pcisel *sel, long reg, int width); void write_config(const struct pcisel *sel, long reg, int width, uint32_t data); +int set_pcir_handler(struct passthru_softc *sc, int reg, int len, + cfgread_handler rhandler, cfgwrite_handler whandler);