FreeBSD 11.x fails to boot w/ SAS3 4Kn HDD and LSISAS3008 on SuperMicro X10DRH-iT

[Rick Miller]

Hi all,

Thanks in advance to anyone that might be able to help. I subscribed to
freebsd-geom@ so that the list did not need to "reply-all". Having trouble
getting FreeBSD 11-STABLE @ r329011 to boot from SAS3 4Kn HDDs via LSISAS
3008 HBA on a SuperMicro X10DRH-iT motherboard after an apparent
installation. All internal media (including all other disks attached to the
HBA) were removed to eliminate other storage being the reason the system
won't boot. This occurs specifically in CSM mode, but the preference is to
boot via UEFI mode instead.

Anyway...booting the machine via the memstick image demonstrates the LSISAS
3008 controller attaching via mpr(4) (whose manpage describes the
controller being supported[1]):

mpr0 at pci0:3:0:0: class=0x010700 card=0x080815d9 chip=0x00971000 rev=0x02
hdr=0x00
  vendor   = 'LSI Logic / Symbios Logic'
  device   = 'SAS3008 PCI-Express Fusion-MPT SAS-3'
  class     = mass storage
  subclass = SAS

The only inserted disk attaches as da0 as illustrated by dmesg:

ses0: pass0,da0: Elemne t descriptor: 'Slot00'
da0 at mpr0 bus 0 scbus0 target 8 lun 0
ses0: pass0,da0: SAS Device Slot Element: 1 Phys at Slot 0
ses0:  phy 0: SAS device type 1 id 0
ses0:  phy 0: protocols: initiator( None ) Target( SSP )
ses0:  phy 0: parent 500304801e870bff addr 5000c500a012814d
da0:  <SEAGATE ST2000NX0263 K004> Fixed Direct Access SPC-4 SCSI device
da0:  Serial Number $serial_number
da0: 1200.000MB/s transfers
da0: Command queueing enabled
da0: 1907729MB (488378648 4096 byte sectors)

The original goal was to boot via zfs root, but when that failed,
subsequent installations used the "Auto (UFS) option" to partition the
disk. For example, the first installation gpart'd the disk as:

# gpart show da0
=>            6  488378635  da0  GPT   (1.8T)
                6              128      1  freebsd-boot  (512K)
            134  487325568      2  freebsd-ufs  (1.8T)
487325702      1048576      3  freebsd-swap  (4.0G)
488374278            4363          - free -   (17M)

The result was a reboot loop. When the system reached the point of reading
the disk, it just rebooted and continued doing so. There was no loader or
beastie menu. Thus, thinking that it could be the partition layout
requirements of the 4Kn disks, it was gpart'd like the below[2][3]. This
was done by exiting to the shell during the partition phase of bsdinstall
and manually gpart'ing the disk according to the below, mounting da0p2 at
/mnt and placing an fstab at /tmp/bsdinstall_etc/fstab that included mount
entries for /dev/da0p2 at / and /dev/da0p3 as swap.

# gpart show da0
=>            6  488378635  da0  GPT   (1.8T)
                6                34          - free -  (136K)
              40              512      1  freebsd-boot  (2.0M)
            552  419430400      2  freebsd-ufs  (1.6T)
419430952      1048576      3  freebsd-swap  (4.0G)
420479528    67899113          - free -   (259G)

When configured as such, the system rebooted at the completion of the
install and appeared to roll through the boot order, which specifies the
HDD first, then CD/DVD, then network. It did attempt to boot via network,
but is irrelevant here.

All the hardware is alleged to be supported by FreeBSD as best I can tell
and OS installation apparently works. I'm at a loss as to why the OS won't
boot. Does someone have feedback or input that may expose why it doesn't
boot?

FWIW, a RHEL7 install was also attempted, which also does not boot.

[1]
https://www.freebsd.org/cgi/man.cgi?query=mpr&apropos=0&sektion=4&manpath=FreeBSD+11.1-RELEASE&arch=default&format=html
[2] https://lists.freebsd.org/pipermail/freebsd-hardware/
2013-September/007380.html
[3] http://www.wonkity.com/~wblock/docs/html/disksetup.html

-- 
Take care
Rick Miller