Server doesn't boot when 3 PCIe slots are populated

[Grzegorz Junka]

On 16/01/2018 05:27, Polytropon wrote:
> On Mon, 15 Jan 2018 22:30:18 +0000, Grzegorz Junka wrote:
>> I tried a different pair of NVMe cards (different adapters with
>> different SSD disks) and the result was exactly the same. Note, that the
>> pair that I tried was previously working in this motherboard without
>> problems for many months, so it's safe to assume that the addition of
>> the network card is causing this problem. But then again, the network
>> card with one of the NVMe drives works fine too.
>>
>> Could be that all three cause some sort of impedance mismatch but that's
>> kind of hard to believe - these are simple cards, there is almost no
>> circuits on the NVMe adapter and the network card is just a chipset with
>> 4 slots.
> No idea if it still applies, but:
>
> I don't know if this idea has come uo yet, but many _many_ years
> ago, I was in a comparable situation. This was of course "traditional
> PCI times" where a common system board had 4 - 6 PCI slots. In that
> particular system, 2 NICs, an ATA "controller", a TV card, and a
> sound card (5 PCI cards total + 1 AGP graphics card) were installed.
> The problem was that the system wouldn't start booting after BIOS POST.
> The solution was to re-arrange cards, followed by a PCI configuration
> reset in the CMOS setup, followed by another boot attempt. In one
> specific card configuration, the system worked as expected, and
> FreeBSD (at that time, probably v4 or v5) would detect all the cards
> without any problems and attach the appropriate drivers. Removing
> one of the cards, or exchanging card positions (for "more convenient
> cabling") would render the system non-booting again. Back to the one
> configuration that worked - system booted. Let me emphasize that it
> was required to reset the PCI configuration in the CMOS setup every
> time such a card change was made, as returning to the "verified
> locations" without doing so would _not_ let the system boot properly.
>
> I'm not sure if it's still that "easy" with modern hardware, though.
> But maybe you can try... :-)

I did try different cards in different slots kind of thing but I didn't 
think about resetting CMOS after changing each configuration. Thanks for 
the suggestion, will try that.
GregJ