svn commit: r348781 - head/usr.sbin/bhyve

[Chuck Tuffli]

Author: chuck
Date: Fri Jun  7 17:09:49 2019
New Revision: 348781
URL: https://svnweb.freebsd.org/changeset/base/348781

Log:
  bhyve: Add PCIe Integrated Endpoint capability
  
  The NVMe CAM driver reports the PCIe Link Capability and Status for
  devices. For emulated bhyve NVMe devices, this looks like:
  
  nda0: nvme version 1.3 x63 (max x63) lanes PCIe Gen15 (max Gen15) link
  
  The driver outputs this because the emulated device doesn't include the
  PCIe Capability structure. The NVMe specification requires these
  registers, so the fix is to add this set of capability registers to the
  emulated device.
  
  Note that PCI Express devices that are integrated into the Root Complex
  (i.e. Bus 0x0) do not have to support the Link Capability or Status
  registers. Windows will fail to start (i.e. Code 10) devices that appear
  to be part of the Root Complex but report being a PCI Express Endpoint.
  So also add a check to pci_emul_add_pciecap() to check if the device is
  integrated and change the device type.
  
  Reviewed by:	imp, ken, araujo, jhb, rgrimes
  Approved by:	imp (mentor), ken (mentor), jhb (maintainer)
  MFC after:	1 week
  Differential Revision: https://reviews.freebsd.org/D19904

Modified:
  head/usr.sbin/bhyve/pci_emul.c
  head/usr.sbin/bhyve/pci_nvme.c

Modified: head/usr.sbin/bhyve/pci_emul.c
==============================================================================
--- head/usr.sbin/bhyve/pci_emul.c	Fri Jun  7 17:05:58 2019	(r348780)
+++ head/usr.sbin/bhyve/pci_emul.c	Fri Jun  7 17:09:49 2019	(r348781)
@@ -953,15 +953,23 @@ pci_emul_add_pciecap(struct pci_devinst *pi, int type)
 	int err;
 	struct pciecap pciecap;
 
-	if (type != PCIEM_TYPE_ROOT_PORT)
-		return (-1);
-
 	bzero(&pciecap, sizeof(pciecap));
 
+	/*
+	 * Use the integrated endpoint type for endpoints on a root complex bus.
+	 *
+	 * NB: bhyve currently only supports a single PCI bus that is the root
+	 * complex bus, so all endpoints are integrated.
+	 */
+	if ((type == PCIEM_TYPE_ENDPOINT) && (pi->pi_bus == 0))
+		type = PCIEM_TYPE_ROOT_INT_EP;
+
 	pciecap.capid = PCIY_EXPRESS;
-	pciecap.pcie_capabilities = PCIECAP_VERSION | PCIEM_TYPE_ROOT_PORT;
-	pciecap.link_capabilities = 0x411;	/* gen1, x1 */
-	pciecap.link_status = 0x11;		/* gen1, x1 */
+	pciecap.pcie_capabilities = PCIECAP_VERSION | type;
+	if (type != PCIEM_TYPE_ROOT_INT_EP) {
+		pciecap.link_capabilities = 0x411;	/* gen1, x1 */
+		pciecap.link_status = 0x11;		/* gen1, x1 */
+	}
 
 	err = pci_emul_add_capability(pi, (u_char *)&pciecap, sizeof(pciecap));
 	return (err);

Modified: head/usr.sbin/bhyve/pci_nvme.c
==============================================================================
--- head/usr.sbin/bhyve/pci_nvme.c	Fri Jun  7 17:05:58 2019	(r348780)
+++ head/usr.sbin/bhyve/pci_nvme.c	Fri Jun  7 17:09:49 2019	(r348781)
@@ -1925,6 +1925,12 @@ pci_nvme_init(struct vmctx *ctx, struct pci_devinst *p
 		goto done;
 	}
 
+	error = pci_emul_add_pciecap(pi, PCIEM_TYPE_ROOT_INT_EP);
+	if (error) {
+		WPRINTF(("%s pci add Express capability failed\r\n", __func__));
+		goto done;
+	}
+
 	pthread_mutex_init(&sc->mtx, NULL);
 	sem_init(&sc->iosemlock, 0, sc->ioslots);