From nobody Thu Aug 25 17:31:41 2022 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MD94Y3bjMz4b3BY; Thu, 25 Aug 2022 17:31:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MD94Y2n71z3LJ6; Thu, 25 Aug 2022 17:31:41 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1661448701; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=G1y9q548Ffwt/mp0QBjFWGnrT0+8TfR6vLbyt5FHhYo=; b=yxmuGzi1PnQjJ3U3S/GkbQlB4ZkdXIYlRi4u6DbhtwkCe/kiouHC08bwy7F5n3EUMmq7/M ynHpYWP/LEnxTSESNnMkBybHRBwSlEIbDp6S5ZfPjzfuunYoJ2LKCmkc3Ii5R0bnqyMOxh FiXk/R55gCP9lNnxy6uxsW6ddwSQnfZo/lxM7oVZSdU7kTgM+uH1uNqoN7B49ADl2GCCMV dNi4SOxPLaLbTTQq9ysjAPdO374uf9jUAWIWakbJmAHb1SdXGDYS4U+ddJj3NPrpn706Dz zgwjsJDCIqVVBTvxz6KVRs8HfQIfUKO4z7Vn7NAN/HtreFbbB+ODxjWlFfrmAg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4MD94Y12hczwFj; Thu, 25 Aug 2022 17:31:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 27PHVf7A043479; Thu, 25 Aug 2022 17:31:41 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 27PHVfO5043478; Thu, 25 Aug 2022 17:31:41 GMT (envelope-from git) Date: Thu, 25 Aug 2022 17:31:41 GMT Message-Id: <202208251731.27PHVfO5043478@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: John Baldwin Subject: git: e5217bedc964 - stable/13 - bhyve: Support other schemes for naming pass-through devices. List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: e5217bedc964639ded1c8f5401012516c8558e39 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1661448701; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=G1y9q548Ffwt/mp0QBjFWGnrT0+8TfR6vLbyt5FHhYo=; b=kDwB1cub0j+hNhEEyAAekAyrUED/agG/Pmsx9KPxSB4fEJmnJ6PA7+0LOxbhpDDB05cnFi hV8F0agu7rR4TJr7eqFCV3h+p8FGB9h+5t/0fWE1W9r+Twugppmfw/Qi7G8qQJsf9Qe/oy q2c1p4/1xwba5tx9eKkKd3i7fovLtctn1UiixihHktMEyF+EIofLrLY/1wRs7a4wFb83WT 2T0wubTuzn5Y8GHvCs5GZsSeXfkjpk8NFtxY73gDvQ3uFsNQ7pZhbPpESpeaqAGK/WuiJa dmuCpDDdx/nXnNa+bcnjZiqBR3J/QuhGlbpZrDcYsaICsidCnbylVzz5qM0GVg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1661448701; a=rsa-sha256; cv=none; b=ObJBziFP4vtucMy4Exkrx31TXz1EVhukK+20KCGUm73QxmmD3b2MJBmvDSfDYHnQjK/pCB iA0NbZIsvwiXrjP/Qh4kRBRv606nlJrmijG3BZECRaY7AMOZEhk2W4vUetshfIB/i2agR1 cW6asnvy2Up7AWxXbQDsMeC9nwTxda+GYci7OKasWT+yxZ2q08R9uzpWtGk5y9gPpWDtUK tHrddXXMGUKtjboZeBmM8hlN2JHbLA7sSvPZwLYvr0FVfxQPSJNoijmQZ/wTogWPzuLi9l DV8UW17dQnmtn3iy+wILRxbFhWOpvY4vgkwBLTpYBLDXAoVxX7BNuDK8Rxi93g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=e5217bedc964639ded1c8f5401012516c8558e39 commit e5217bedc964639ded1c8f5401012516c8558e39 Author: John Baldwin AuthorDate: 2022-08-19 21:55:29 +0000 Commit: John Baldwin CommitDate: 2022-08-25 16:38:33 +0000 bhyve: Support other schemes for naming pass-through devices. Permit naming pass through devices using the syntax accepted by pciconf (pci[:]::) as well as by device name (e.g. "ppt0"). While here, fix an error in the manpage that had the bus and slot arguments for the original /-delimited scheme swapped. Reviewed by: imp, markj Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D36147 (cherry picked from commit baf753cc196e2308828c7fb9b0385d5fe17ecdc2) --- usr.sbin/bhyve/bhyve.8 | 27 ++++++++-- usr.sbin/bhyve/bhyve_config.5 | 12 ++++- usr.sbin/bhyve/pci_passthru.c | 112 ++++++++++++++++++++++++++++++++++++------ 3 files changed, 131 insertions(+), 20 deletions(-) diff --git a/usr.sbin/bhyve/bhyve.8 b/usr.sbin/bhyve/bhyve.8 index a352a7ba2a93..1de8c516b785 100644 --- a/usr.sbin/bhyve/bhyve.8 +++ b/usr.sbin/bhyve/bhyve.8 @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd October 13, 2021 +.Dd August 19, 2022 .Dt BHYVE 8 .Os .Sh NAME @@ -367,6 +367,7 @@ Network device backends: .Xc .El .Sm on +.Pp If .Cm mac is not specified, the MAC address is derived from a fixed OUI and the @@ -492,14 +493,32 @@ in the guest address space reserved for boot firmware. .El .Pp Pass-through device backends: -.Bl -tag -width 10n -.It Ns Ar slot Ns Cm \&/ Ns Ar bus Ns Cm \&/ Ns Ar function -Connect to a PCI device on the host at the selector described by +.Sm off +.Bl -bullet +.It +.Cm ppt Ar N Oo , Ar passthru-device-options Oc +.It +.Ns Ar bus Cm \&/ Ar slot Cm \&/ Ar function +.Op , Ar passthru-device-options +.It +.Cm pci Ar bus Cm : Ar slot Cm : Ns Ar function +.Op , Ar passthru-device-options +.El +.Sm on +.Pp +Connect to a PCI device on the host either named ppt +.Ns Ar N +or at the selector described by .Ar slot , .Ar bus , and .Ar function numbers. +.Pp +The +.Ar passthru-device-options +are: +.Bl -tag -width 10n .It Cm rom= Ns Ar romfile Add .Ar romfile diff --git a/usr.sbin/bhyve/bhyve_config.5 b/usr.sbin/bhyve/bhyve_config.5 index fbbea50bc7be..b4160a37620d 100644 --- a/usr.sbin/bhyve/bhyve_config.5 +++ b/usr.sbin/bhyve/bhyve_config.5 @@ -23,7 +23,7 @@ .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" -.Dd September 17, 2021 +.Dd August 19, 2022 .Dt BHYVE_CONFIG 5 .Os .Sh NAME @@ -538,6 +538,12 @@ If set, allocate a memory disk as the backing store. The value of this variable is the size of the memory disk in megabytes. .El .Ss PCI Passthrough Settings +The +.Xr ppt 4 +device driver must be attached to the +PCI device being passed through. +The device to pass through can be identified either by name or its +host PCI bus location. .Bl -column "Name" "integer" "Default" .It Sy Name Ta Sy Format Ta Sy Default Ta Sy Description .It Va bus Ta integer Ta Ta @@ -546,6 +552,10 @@ Host PCI bus address of device to pass through. Host PCI slot address of device to pass through. .It Va func Ta integer Ta Ta Host PCI function address of device to pass through. +.It Va pptdev Ta string Ta Ta +Name of a +.Xr ppt 4 +device to pass through. .It Va rom Ta path Ta Ta ROM file of the device which will be executed by OVMF to init the device. .El diff --git a/usr.sbin/bhyve/pci_passthru.c b/usr.sbin/bhyve/pci_passthru.c index 0dfaf5908670..9940a13250ac 100644 --- a/usr.sbin/bhyve/pci_passthru.c +++ b/usr.sbin/bhyve/pci_passthru.c @@ -52,6 +52,7 @@ __FBSDID("$FreeBSD$"); #ifndef WITHOUT_CAPSICUM #include #endif +#include #include #include #include @@ -132,7 +133,7 @@ pcifd_init(void) errx(EX_OSERR, "Unable to apply rights for sandbox"); const cap_ioctl_t pcifd_ioctls[] = { PCIOCREAD, PCIOCWRITE, PCIOCGETBAR, - PCIOCBARIO, PCIOCBARMMAP }; + PCIOCBARIO, PCIOCBARMMAP, PCIOCGETCONF }; if (caph_ioctls_limit(pcifd, pcifd_ioctls, nitems(pcifd_ioctls)) == -1) errx(EX_OSERR, "Unable to apply rights for sandbox"); #endif @@ -652,30 +653,39 @@ done: static int passthru_legacy_config(nvlist_t *nvl, const char *opts) { + const char *cp; + char *tofree; char value[16]; int bus, slot, func; if (opts == NULL) return (0); - if (sscanf(opts, "%d/%d/%d", &bus, &slot, &func) != 3) { + cp = strchr(opts, ','); + + if (strncmp(opts, "ppt", strlen("ppt")) == 0) { + tofree = strndup(opts, cp - opts); + set_config_value_node(nvl, "pptdev", tofree); + free(tofree); + } else if (sscanf(opts, "pci0:%d:%d:%d", &bus, &slot, &func) == 3 || + sscanf(opts, "pci%d:%d:%d", &bus, &slot, &func) == 3 || + sscanf(opts, "%d/%d/%d", &bus, &slot, &func) == 3) { + snprintf(value, sizeof(value), "%d", bus); + set_config_value_node(nvl, "bus", value); + snprintf(value, sizeof(value), "%d", slot); + set_config_value_node(nvl, "slot", value); + snprintf(value, sizeof(value), "%d", func); + set_config_value_node(nvl, "func", value); + } else { EPRINTLN("passthru: invalid options \"%s\"", opts); return (-1); } - snprintf(value, sizeof(value), "%d", bus); - set_config_value_node(nvl, "bus", value); - snprintf(value, sizeof(value), "%d", slot); - set_config_value_node(nvl, "slot", value); - snprintf(value, sizeof(value), "%d", func); - set_config_value_node(nvl, "func", value); - - opts = strchr(opts, ','); - if (opts == NULL) { + if (cp == NULL) { return (0); } - return pci_parse_legacy_config(nvl, opts + 1); + return (pci_parse_legacy_config(nvl, cp + 1)); } static int @@ -729,6 +739,72 @@ passthru_init_rom(struct vmctx *const ctx, struct passthru_softc *const sc, return (0); } +static bool +passthru_lookup_pptdev(const char *name, int *bus, int *slot, int *func) +{ + struct pci_conf_io pc; + struct pci_conf conf[1]; + struct pci_match_conf patterns[1]; + char *cp; + + bzero(&pc, sizeof(struct pci_conf_io)); + pc.match_buf_len = sizeof(conf); + pc.matches = conf; + + bzero(&patterns, sizeof(patterns)); + + /* + * The pattern structure requires the unit to be split out from + * the driver name. Walk backwards from the end of the name to + * find the start of the unit. + */ + cp = strchr(name, '\0'); + assert(cp != NULL); + while (cp != name && isdigit(cp[-1])) + cp--; + if (cp == name || !isdigit(*cp)) { + EPRINTLN("Invalid passthru device name %s", name); + return (false); + } + if ((size_t)(cp - name) + 1 > sizeof(patterns[0].pd_name)) { + EPRINTLN("Passthru device name %s is too long", name); + return (false); + } + memcpy(patterns[0].pd_name, name, cp - name); + patterns[0].pd_unit = strtol(cp, &cp, 10); + if (*cp != '\0') { + EPRINTLN("Invalid passthru device name %s", name); + return (false); + } + patterns[0].flags = PCI_GETCONF_MATCH_NAME | PCI_GETCONF_MATCH_UNIT; + pc.num_patterns = 1; + pc.pat_buf_len = sizeof(patterns); + pc.patterns = patterns; + + if (ioctl(pcifd, PCIOCGETCONF, &pc) == -1) { + EPRINTLN("ioctl(PCIOCGETCONF): %s", strerror(errno)); + return (false); + } + if (pc.status != PCI_GETCONF_LAST_DEVICE && + pc.status != PCI_GETCONF_MORE_DEVS) { + EPRINTLN("error returned from PCIOCGETCONF ioctl"); + return (false); + } + if (pc.num_matches == 0) { + EPRINTLN("Passthru device %s not found", name); + return (false); + } + + if (conf[0].pc_sel.pc_domain != 0) { + EPRINTLN("Passthru device %s on unsupported domain", name); + return (false); + } + *bus = conf[0].pc_sel.pc_bus; + *slot = conf[0].pc_sel.pc_dev; + *func = conf[0].pc_sel.pc_func; + return (true); +} + static int passthru_init(struct vmctx *ctx, struct pci_devinst *pi, nvlist_t *nvl) { @@ -758,9 +834,15 @@ passthru_init(struct vmctx *ctx, struct pci_devinst *pi, nvlist_t *nvl) var = atoi(value); \ } while (0) - GET_INT_CONFIG(bus, "bus"); - GET_INT_CONFIG(slot, "slot"); - GET_INT_CONFIG(func, "func"); + value = get_config_value_node(nvl, "pptdev"); + if (value != NULL) { + if (!passthru_lookup_pptdev(value, &bus, &slot, &func)) + return (error); + } else { + GET_INT_CONFIG(bus, "bus"); + GET_INT_CONFIG(slot, "slot"); + GET_INT_CONFIG(func, "func"); + } if (vm_assign_pptdev(ctx, bus, slot, func) != 0) { warnx("PCI device at %d/%d/%d is not using the ppt(4) driver",