AOC-S3008L-L8i - OS Volume Not Recognized After Upgrade

From: Jason Breitman <jbreitman_at_tildenparkcapital.com>
Date: Wed, 26 Oct 2022 14:07:44 UTC
Please review the details below and let me know if there is a setting that I should apply to my FreeBSD Server or if there is a bug fix that I can apply to resolve my issue.

Issue
I am unable to see the RAID 1 operating system volume after upgrading from FreeBSD 12.3 to 13.1.
The LSI Logical Volume 3000 does not appear via camcontrol devlist and or the sas3ircu utility after the upgrade to FreeBSD 13.1.
I am able to see the 3 LSI controllers after the upgrade, but not the volume or drives used for the operating system.

I was able to rollback via freebsd-update rollback and am now able to boot the system again.
I can boot off of a FreeBSD 13.1 ISO and run into the same issue of not being able to see the RAID 1 volume.


Environment
FreeBSD 12.3-RELEASE-p6 FreeBSD 12.3-RELEASE-p6 GENERIC  amd64

Supermicro AOC-S3008L-L8i SAS Controller 8-port internal (2x SFF-8643) SAS3 12Gb/s
# sas3flash -list -c 2
Avago Technologies SAS3 Flash Utility
Version 17.00.00.00 (2018.04.02) 
Copyright 2008-2018 Avago Technologies. All rights reserved.

	Adapter Selected is a Avago SAS: SAS3008(C0)

	Controller Number              : 2
	Controller                     : SAS3008(C0)
	PCI Address                    : 00:af:00:00
	SAS Address                    : 5003048-0-22ed-0af0
	NVDATA Version (Default)       : 0e.01.31.27
	NVDATA Version (Persistent)    : 0e.01.31.27
	Firmware Product ID            : 0x2721 (IR)
	Firmware Version               : 16.00.10.00
	NVDATA Vendor                  : LSI
	NVDATA Product ID              : LSI3008-IR
	BIOS Version                   : 08.37.00.00
	UEFI BSD Version               : 18.00.00.00
	FCODE Version                  : N/A
	Board Name                     : LSI3008-IR
	Board Assembly                 : N/A
	Board Tracer Number            : N/A

	Finished Processing Commands Successfully.
	Exiting SAS3Flash.


Error Message

Trying to mount root from ufs: /dev/da12p2 [rw]
mountroot: waiting for device /dev/da12p2
Mounting from ufs: /dev/da12p2 failed with error 19.

Loader variables:
   vfs.root.mountfrom=ufs: /dev/da12p2
   vfs.root.mountfrom.options=rw


Below is the output from several commands on the server when running FreeBSD 12.3.
# camcontrol devlist | grep LSI
<LSI Logical Volume 3000>          at scbus2 target 0 lun 0 (pass12,da12)

# pciconf -lv | grep -A 3 mpr
mpr0@pci0:94:0:0:	class=0x010700 card=0x080815d9 chip=0x00971000 rev=0x02 hdr=0x00
    vendor     = 'Broadcom / LSI'
    device     = 'SAS3008 PCI-Express Fusion-MPT SAS-3'
    class      = mass storage
--
mpr1@pci0:95:0:0:	class=0x010700 card=0x080815d9 chip=0x00971000 rev=0x02 hdr=0x00
    vendor     = 'Broadcom / LSI'
    device     = 'SAS3008 PCI-Express Fusion-MPT SAS-3'
    class      = mass storage
--
mpr2@pci0:175:0:0:	class=0x010700 card=0x080815d9 chip=0x00971000 rev=0x02 hdr=0x00
    vendor     = 'Broadcom / LSI'
    device     = 'SAS3008 PCI-Express Fusion-MPT SAS-3'
    class      = mass storage

# sas3ircu 2 DISPLAY
Avago Technologies SAS3 IR Configuration Utility.
Version 11.00.00.00 (2015.08.04) 
Copyright (c) 2009-2015 Avago Technologies. All rights reserved. 

Read configuration has been initiated for controller 2
------------------------------------------------------------------------
Controller information
------------------------------------------------------------------------
  Controller type                         : SAS3008
  PI Supported                            : Yes
  PI Mixing                               : Disabled
  BIOS version                            : 8.37.00.00
  Firmware version                        : 16.00.10.00
  Channel description                     : 1 Serial Attached SCSI
  Initiator ID                            : 0
  Maximum physical devices                : 255
  Concurrent commands supported           : 3072
  Slot                                    : 1
  Segment                                 : 0
  Bus                                     : 175
  Device                                  : 0
  Function                                : 0
  RAID Support                            : Yes
------------------------------------------------------------------------
IR Volume information
------------------------------------------------------------------------
IR volume 1
  Volume ID                               : 323
  PI Supported                            : No
  Status of volume                        : Okay (OKY)
  Volume wwid                             : 024ccc5729aa3be7
  RAID level                              : RAID1
  Size (in MB)                            : 456809
  Boot                                    : Primary
  Physical hard disks                     :
  PHY[0] Enclosure#/Slot#                 : 1:0
  PHY[1] Enclosure#/Slot#                 : 1:1
------------------------------------------------------------------------
Physical device information
------------------------------------------------------------------------
Initiator at ID #0

Device is a Hard disk
  Enclosure #                             : 1
  Slot #                                  : 0
  PI Supported                            : No
  SAS Address                             : 4433221-1-0000-0000
  State                                   : Optimal (OPT)
  Size (in MB)/(in sectors)               : 457862/937703087
  Manufacturer                            : ATA     
  Model Number                            : Micron_5300_MTFD
  Firmware Revision                       : U001
  Serial No                               : 20182841484F
  GUID                                    : 500a07512841484f
  Protocol                                : SATA
  Drive Type                              : SATA_SSD

Device is a Hard disk
  Enclosure #                             : 1
  Slot #                                  : 1
  PI Supported                            : No
  SAS Address                             : 4433221-1-0100-0000
  State                                   : Optimal (OPT)
  Size (in MB)/(in sectors)               : 457862/937703087
  Manufacturer                            : ATA     
  Model Number                            : Micron_5300_MTFD
  Firmware Revision                       : U001
  Serial No                               : 20182841489A
  GUID                                    : 500a07512841489a
  Protocol                                : SATA
  Drive Type                              : SATA_SSD
------------------------------------------------------------------------
Enclosure information
------------------------------------------------------------------------
  Enclosure#                              : 1
  Logical ID                              : 50030480:22ed0af0
  Numslots                                : 8
  StartSlot                               : 0
------------------------------------------------------------------------
SAS3IRCU: Command DISPLAY Completed Successfully.
SAS3IRCU: Utility Completed Successfully.


Jason Breitman