Re: Can't upgrade 13.3->13.4, kernel hangs at launch

On 1/27/25 03:09, list_freebsd@bluerosetech.com wrote:
> For some reason I missed the rest of your email when replying earlier. 
> Replies to the remainder below.
> 
> tl;dr: Solved.  It was the CPU microcode updater
> 
> On 2025-01-25 10:48, Edward Sanford Sutton, III wrote:
>> On 1/24/25 07:03, list_freebsd@bluerosetech.com wrote:
>>> Watching the console, the pre-graphical part goes by too quickly for me
>>> to read, but I do catch a big box made of asterisks that looks like the
>>> one shown in a forum thread[1] about this issue.  When I get to the
>>> graphical boot menu, it has the "Loader needs to be updated" warning
>>> just above the autoboot countdown.
>>
>>     If you have access to a video camera (even a cell phone with it) then
>> you could make a recording that you could load into a player to let you
>> stop on the correct frames. Many players let you slow down the video a
>> lot and navigate one frame at a time to make it easier to hit that short
>> moment or else ffmpeg could turn the video into separate jpg images for
>> review.
> 
> IPMI console recorder ftw. :)
> 
> It's what I thought it was: the big, shouty, asterisk-framed update 
> warning.  Loader output otherwise normal.
> 
>>     What does your /boot/loader.conf look like? Is your UEFI/BIOS up to
>> date? I'm thinking some kind of kernel module incompatibility/bug,
>> kernel bug, motherboard UEFI bug, or hardware issue.
> 
> It appears there is a problem with the microcode updater 
> (intel-20241112).  This makes the 13.4 kernel hang:
> 
> # grep '^[^#]' /tmp/BE13.3/boot/loader.conf
> cpu_microcode_load="YES"
> cpu_microcode_name="/boot/firmware/intel-ucode.bin"
> geom_eli_load="YES"
> zfs_load="YES"
> kern.geom.label.disk_ident.enable=0
> kern.geom.label.gptid.enable=0
> 
> This does not:
> 
> # grep '^[^#]' /boot/loader.conf
> geom_eli_load="YES"
> zfs_load="YES"
> kern.geom.label.disk_ident.enable=0
> kern.geom.label.gptid.enable=0
> 
> The hardware is a Xeon E3-1245v6 on a Supermicro X11SSH-LN4F mainboard.

   Diagnosing this goes beyond my abilities but sounds like a good 
candidate for a formal PR. It may be useful to determine if a crash is 
reproducible if applying the microcode later during boot 
(sysutils/cpu-microcode-rc).
   If building a custom kernel/world is needed to further diagnose, it 
can be a bit time consuming (the build itself) but is otherwise not 
difficult. Handbook has good general steps and anyone needing changes 
can likely guide you through what small configure changes to make 
according to the needs. I'd be prepared and willing to share system 
details (that cpu model, motherboard+bios version, etc.; sysutils/cpu-x 
can help gather it further from boot logs, sysctl probes, etc.) and 
system logs, crash dumps, and back traces and it may be wise to open a 
formal PR. I haven't found existing reports sounding similar in my 
searching.