Server doesn't boot when 3 PCIe slots are populated

Mehmet Erol Sanliturk m.e.sanliturk at
Sun Jan 14 16:18:11 UTC 2018

On Sun, Jan 14, 2018 at 5:46 PM, Grzegorz Junka <list1 at> wrote:

> On 13/01/2018 17:56, Mehmet Erol Sanliturk wrote:
>> On Sat, Jan 13, 2018 at 7:21 PM, Grzegorz Junka <list1 at
>> <mailto:list1 at>> wrote:
>>     Hello,
>>     I am installing a FreeBSD server based on Supermicro H8SML-iF.
>>     There are three PCIe slots to which I installed 2 NVMe drives and
>>     one network card Intel I350-T4 (with 4 Ethernet slots).
>>     I am observing a strange behavior where the system doesn't boot if
>>     all three PCIe slots are populated. It shows this message:
>>     nvme0: <Generic NVMe Device> mem 0xfd8fc000-0xfd8fffff irq 24 at
>>     device 0.0 on pci1
>>     nvme0: controller ready did not become 1 within 30000 ms
>>     nvme0: did not complete shutdown within 5 seconds of notification
>>     The I see a kernel panic/dump and the system reboots after 15 seconds.
>>     If I remove one card, either one of the NVMe drives or the network
>>     card, the system boots fine. Also, if in BIOS I set PnP OS to YES
>>     then sometimes it boots (but not always). If I set PnP OS to NO,
>>     and all three cards are installed, the system never boots.
>>     When the system boots OK I can see that the network card is
>>     reported as 4 separate devices on one of the PCIe slots. I tried
>>     different NVMe drives as well as changing which device is
>>     installed to which slot but the result seems to be the same in any
>>     case.
>>     What may be the issue? Amount of power drawn by the hardware? Too
>>     many devices not supported by the motherboard? Too many interrupts
>>     for the FreeBSD kernel to handle?
>>     Any help would be greatly appreciated.
>>     GregJ
>>     _______________________________________________
>> From my experience from other trade marked main boards , an action may be
>> to check manual of your server board to see whether there are rules about
>> use of these slots : Sometimes differently shaped slots are supplied with
>> same ports : If one slot is occupied , the other slot should be left open ,
>> or rules about not to insert such a kind of device into a slot , for
>> example , graphic cards .
>> Mehmet Erol Sanliturk
> I checked the manual but couldn't find any restrictions regarding PCIe
> ports. It only says how many lanes are available in each slot. Would there
> be any obvious BIOS setting that could cause this issue? I tried after
> resetting BIOS to default settings but maybe something is set incorrectly
> by default?
> GregJ
> _______________________________________________

On the above page , click "OS Compatibility"

On the following page , click "SR5650"
OS Compatibility Chart

On the column ( third )


there listed only

FreeBSD 8.0
FreeBSD 9.1

>From this list , it may be said that , this mother board date is old ,
means , it seems that the new OS versions are not tested after currently
tested OS versions .

To check interaction between operating system and your Supermicro H8SML-iF
, select one of the suitable operating system ( Unix class OSes are more
suitable  ) for you and tested on this card , and try to install it as you
like your installed components . If it boots successfully , it means that
there is an incompatibility between your FreeBSD and the main board . If no
one of them boots , then you may conclude that , there is a problem in your
settings .

BIOS settings are important , because , OS communicates with the main board
through these settings .

In manual ( downloaded from the above page :
Manual Revision 1.0c
Release Date: March 12, 2014 ) , page 4-9  , "PCI/PnP Configuration" is
defined .
If PnP is selected YES. OS adjusts some device settings  . If NO is
selected , BIOS adjusts some device settings . When BIOS adjusted device
settings are not conforming to OS parameters , the result will be "FAIL" .

Therefore , more suitable selection is YES .

Another point is that , there are many more BIOS selectable parameters and
jumpers about PCI slots and others  .
There are some BIOS settings for PCI slots :

PCI X4 Slot 6 ( page 4-9 )
PCI x8 Slot 7 ( page 4-10 )

Please review these BIOS settings in your manual and set them with respect
to your requirements .

Mehmet Erol Sanliturk

More information about the freebsd-questions mailing list