SR-IOV Patch Series 4/7: Core SR-IOV infrastructure in pci driver

[Ryan Stone]

This patch series handles creating VFs, allocating memory space
resources to them, and destroying them.

I have two concerns with these patches:
Patch 8: Do I need to do any special handling for big-endian
architectures when emulating the PCI vendor field?
Patch 9: I'm not very familiar with the rman API and I worry that I
might be abusing the API or using functions that are too low-level for
what I'm trying to accomplish.

Oh, and the reason why I use Giant in these patches is that the PCI
subsystem (and newbus) are currently locked by Giant.  I didn't see
any reasonable way to extricate myself for Giant here, so I punted on
the problem and whoever finally deals with newbus locking will
hopefully be able to fix the problem holistically.


http://people.freebsd.org/~rstone/patches/iov/0007-Implement-interface-to-create-SR-IOV-Virtual-Functio.patch
[PATCH 07/21] Implement interface to create SR-IOV Virtual Functions

Implement the interace to create SR-IOV Virtual Functions (VFs).
When a driver registers that they support SR-IOV by calling
pci_setup_iov(), the SR-IOV code creates a new node in /dev/iov
for that device.  An ioctl can be invoked on that device to
create VFs and have the driver initialize them.

At this point, allocating memory I/O windows (BARs) is not
supported.
---
 sys/amd64/conf/GENERIC        |   1 +
 sys/conf/files                |   1 +
 sys/conf/options              |   1 +
 sys/dev/pci/pci.c             |  29 ++++
 sys/dev/pci/pci_if.m          |  24 +++
 sys/dev/pci/pci_iov.c         | 380 ++++++++++++++++++++++++++++++++++++++++++
 sys/dev/pci/pci_iov_private.h |  40 +++++
 sys/dev/pci/pci_private.h     |   6 +
 sys/dev/pci/pcireg.h          |  18 ++
 sys/dev/pci/pcivar.h          |  22 +++
 sys/i386/conf/GENERIC         |   1 +
 sys/sys/iov.h                 |  43 +++++
 12 files changed, 566 insertions(+)

http://people.freebsd.org/~rstone/patches/iov/0008-Emulate-the-Device-ID-and-Vendor-ID-registers-for-VF.patch
[PATCH 08/21] Emulate the Device ID and Vendor ID registers for VFs

The SR-IOV standard requires VFs to read all-ones when the VID
and DID registers are read.  The VMM (hypervisor) is required to
emulate them instead.  Make pci_read_config() do this emulation.

Change pci_user.c to use pci_read_config() to read config space
registers instead of going directly to the pcib so that the
emulated VID/DID registers work correctly on VFs.  This is
required both for pciconf and bhyte PCI passthrough.
---
 sys/dev/pci/pci.c      | 31 +++++++++++++++++++++++++++++++
 sys/dev/pci/pci_user.c | 20 ++++----------------
 2 files changed, 35 insertions(+), 16 deletions(-)

http://people.freebsd.org/~rstone/patches/iov/0009-Allocate-PCI-I-O-memory-spaces-for-VFs.patch
[PATCH 09/21] Allocate PCI I/O memory spaces for VFs

When creating VFs, we must size each SR-IOV BAR on the PF and
allocate a configuous I/O memory window large enough for every VF.
However, the window only needs to be aligned to a boundary equal
to the size of the window for a single VF.

When a VF attempts to allocate an I/O memory resource, we must
intercept the request in the pci driver and pass it off to the
SR-IOV code, which will allocate the correct window from the
pre-allocated memory space for the PF.

Inform the pci driver about the size and address of the BARs on
the VF when the VF is created.  This is required by pciconf -b and
bhyve.
---
 sys/dev/pci/pci.c             |  35 ++++++++
 sys/dev/pci/pci_iov.c         | 200 +++++++++++++++++++++++++++++++++++++++++-
 sys/dev/pci/pci_iov_private.h |  13 +++
 sys/dev/pci/pci_private.h     |   6 ++
 sys/dev/pci/pcivar.h          |   2 +-
 5 files changed, 252 insertions(+), 4 deletions(-)

http://people.freebsd.org/~rstone/patches/iov/0010-Add-interface-to-destroy-SR-IOV-VFs.patch
[PATCH 10/21] Add interface to destroy SR-IOV VFs

---
 sys/dev/pci/pci_iov.c         | 113 +++++++++++++++++++++++++++++++++++++++++-
 sys/dev/pci/pci_iov_private.h |   1 +
 sys/sys/iov.h                 |   1 +
 3 files changed, 113 insertions(+), 2 deletions(-)