PCIe NVME drives not detected on Dell R6515

[Miroslav Lachman]

On 2020-04-27 08:02, Miroslav Lachman wrote:
> I don't know what is with Scott. I hope he is well.
> Is there somebody else who can help me with this issue?
> Scott wrote there are hotplug PCIe buses not probed during boot process. 
> I am not a developer so I cannot move forward alone.

The problem is with PCIe Hot Plug.
Hot Plug bus was not enumerated thus no NVME detected.

Dan Lukes suggested (privately) to disable hot plugging by this at 
second stage loader prompt:

set hw.pci.enable_pcie_hp=0

Then I was able to boot FreeBSD installer ISO in BIOS mode (I don't know 
why but this machine is not able to boot FreeBSD ISO media in UEFI 
mode). Installer sees both NVME disks and installation was successful 
but it cannot boot - Dell R6515 in BIOS mode does not show NVME drives. 
Switching to UEFI boot shows disks but they didn't contained EFI 
partition boot code.
When I modified the partitions layout (remove swap, ad efi partition and 
swap again) it is now able to boot FreeBSD 11.3 amd64 in UEFI mode from 
NVME disks with Hot Plug disabled in loader.conf.

Can somebody look on to it why the bus is not probed when Hot Plug is 
enabled?

I have a few days to run some tests on this HW before it will go in to 
production.

Kind regards
Miroslav Lachman