Updates to bhyve NVMe emulation

[Chuck Tuffli]

In an effort to provide better NVMe emulation in bhyve, I've started
running the NVM Express Compliance tests against the bhyve emulation
and fixing issues the tests find. The tests are part of the University
of New Hampshire (UNH) Interoperability Lab (IOL) suite.

I have posted a series of patches in Phabricator and would appreciate
any feedback people might have. And because there are a few changes,
the plan is to leave them up for review longer than I normally do.
Note that I've cross-posted to the SCSI list to possibly pickup NVMe
expertise. Summary of the patch series with links to differential
reviews:

bhyve: convert NVMe logging statements
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 157 insertions(+), 157 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24878

bhyve: refactor NVMe IO command handling
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 175 insertions(+), 180 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24879

bhyve: implement NVMe Flush command
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 43 insertions(+), 1 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24880

bhyve: add a comment explaining NVME dsm option
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 2 insertions(+), 1 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24881

bhyve: add locks around NVMe queue accesses
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 104 insertions(+), 61 deletions(-)
Differential Revision: https://reviews.freebsd.org/D19841

bhyve: Consolidate NVMe CQ update
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 51 insertions(+), 49 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24882

bhyve: implement NVMe Namespace Identification Descriptor
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 20 insertions(+), 11 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24884

bhyve: fix NVMe Get Log Page command
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 15 insertions(+), 5 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24885

bhyve: fix NVMe queue creation and deletion
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 89 insertions(+), 34 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24886

bhyve: add more compliant NVMe Get/Set Features
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 214 insertions(+), 152 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24887

bhyve: make unsupported NVMe commands a debug message
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 1 insertions(+), 1 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24888

bhyve: implement NVMe Format NVM command
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 78 insertions(+), 0 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24889

bhyve: refactor NVMe I/O read/write
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 108 insertions(+), 78 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24890

bhyve: base pci_nvme_ioreq size on advertised MDTS
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 60 insertions(+), 92 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24891

bhyve: validate NVMe deallocate range values
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 9 insertions(+), 0 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24893

bhyve: implement NVMe SMART data I/O statistics
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 71 insertions(+), 1 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24894

bhyve: validate the NVMe LBA start and count
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 54 insertions(+), 15 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24895

bhyve: Add AER support to NVMe emulation
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 115 insertions(+), 6 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24896

bhyve: add basic NVMe Firmware Commit support
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 9 insertions(+), 0 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24897

bhyve: add NVMe Feature Interrupt Vector Config
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 39 insertions(+), 0 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24898

bhyve: fix NVMe Get Features, Predictable Latency
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 4 insertions(+), 0 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24899

bhyve: NVMe handle zero length DSM ranges
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 45 insertions(+), 33 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24900

bhyve: fix NVMe Active Namespace list
 usr.sbin/bhyve/pci_nvme.c
 1 files changed, 2 insertions(+), 1 deletions(-)
Differential Revision: https://reviews.freebsd.org/D24901

--chuck