Stable/13 doesn't boot with xen

From: Brian Buhrow <buhrow_at_nfbcal.org>
Date: Sat, 04 Jun 2022 20:03:55 UTC
	Hello.  In the process of getting the netback.c patch into a working kernel on the system
I'm updating, I discovered that Stable/13 doesn't boot successfully when xen-4.16 is running.
Specifically, the system hangs while configuring the PCI busses.  I suspect it has something to
do with the failure of the hpet timers to attach.
I do not know if it works with older versions of the xen hypervisor, but it boots and runs fine
without xen running.  Below are two boot logs, the first with xen installed and running, the
one that fails, and the successfful boot,  with FreeBSD running on bare metal.
Hopefully this is a known issue and there is an easy fix.

-thanks
-Brian

Version info:

FreeBSD xen-lothlorien.nfbcal.org 13.1-STABLE FreeBSD 13.1-STABLE #0 stable/13-984a45d77: Sat Jun  4 07:38:09 PDT 2022     buhrow@fbsd_dev.nfbcal.org:/usr/home/buhrow/obj/usr/home/buhrow/src/fbsd-src/13/amd64.amd64/sys/GENERIC amd64

<broken boot log with xen>

BIOS drive C: is disk0
BIOS drive D: is disk1
BIOS drive E: is disk2
BIOS drive F: is disk3

FreeBSD/x86 bootstrap loader, Revision 1.1
Loading /boot/defaults/loader.conf
Loading /boot/defaults/loader.conf
Loading /boot/device.hints
Loading /boot/loader.conf
Loading /boot/loader.conf.local
Autoboot in 1 seconds. [Space] to pause 
Loading Xen kernel...
/boot/xen data=0x2659c8+0x13e638 -
Loading kernel...
/boot/kernel/kernel 
Loading configured modules...
/boot/kernel/ipmi.ko size 0x11950 at 0x21a6000
loading required module 'smbus'
/boot/kernel/smbus.ko size 0x3cb0 at 0x21b8000
/boot/kernel/geom_mirror.ko size 0x20c80 at 0x21bc000
/boot/kernel/tpm.ko size 0xad70 at 0x21dd000
/boot/firmware/intel-ucode.bin -size=0x303800
/etc/hostid size=0x25
/boot/entropy size=0x1000
 Xen 4.16.0
(XEN) Xen version 4.16.0 (buhrow@) (FreeBSD clang version 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)) debug=n Thu Jun  2 11:30:17 PDT 2022
(XEN) Latest ChangeSet: 
(XEN) Bootloader: FreeBSD Loader
(XEN) Command line: dom0_mem=8192m dom0_max_vcpus=2 dom0=pvh pv-l1tf=off,domu=off console=com1,vga com1=9600,8n1
(XEN) Xen image load base address: 0
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: none; EDID transfer time: 0 seconds
(XEN)  EDID info not retrieved because no DDC retrieval method detected
(XEN) Disc information:
(XEN)  Found 4 MBR signatures
(XEN)  Found 4 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  [0000000000000000, 00000000000997ff] (usable)
(XEN)  [0000000000099800, 000000000009ffff] (reserved)
(XEN)  [00000000000e0000, 00000000000fffff] (reserved)
(XEN)  [0000000000100000, 000000001fffffff] (usable)
(XEN)  [0000000020000000, 00000000201fffff] (reserved)
(XEN)  [0000000020200000, 000000003fffffff] (usable)
(XEN)  [0000000040000000, 00000000401fffff] (reserved)
(XEN)  [0000000040200000, 00000000bc855fff] (usable)
(XEN)  [00000000bc856000, 00000000bc85efff] (ACPI data)
(XEN)  [00000000bc85f000, 00000000bc8a9fff] (ACPI NVS)
(XEN)  [00000000bc8aa000, 00000000bc8b1fff] (usable)
(XEN)  [00000000bc8b2000, 00000000bc9a4fff] (reserved)
(XEN)  [00000000bc9a5000, 00000000bc9a6fff] (usable)
(XEN)  [00000000bc9a7000, 00000000bcbc5fff] (reserved)
(XEN)  [00000000bcbc6000, 00000000bcbc6fff] (usable)
(XEN)  [00000000bcbc7000, 00000000bcbd6fff] (reserved)
(XEN)  [00000000bcbd7000, 00000000bcbf4fff] (ACPI NVS)
(XEN)  [00000000bcbf5000, 00000000bcc18fff] (reserved)
(XEN)  [00000000bcc19000, 00000000bcc5bfff] (ACPI NVS)
(XEN)  [00000000bcc5c000, 00000000bce7bfff] (reserved)
(XEN)  [00000000bce7c000, 00000000bcffffff] (usable)
(XEN)  [00000000bd800000, 00000000bf9fffff] (reserved)
(XEN)  [00000000fed1c000, 00000000fed3ffff] (reserved)
(XEN)  [00000000ff000000, 00000000ffffffff] (reserved)
(XEN)  [0000000100000000, 000000083e5fffff] (usable)
(XEN) New Xen image base address: 0xbc200000
(XEN) ACPI: RSDP 000F0450, 0024 (r2  INTEL)
(XEN) ACPI: XSDT BC856070, 0064 (r1 INTEL  DQ67SW    1072009 AMI     10013)
(XEN) ACPI: FACP BC85DBC0, 00F4 (r4 INTEL  DQ67SW    1072009 AMI     10013)
(XEN) ACPI: DSDT BC856168, 7A54 (r2 INTEL  DQ67SW         16 INTL 20051117)
(XEN) ACPI: FACS BCBDBF80, 0040
(XEN) ACPI: APIC BC85DCB8, 0072 (r3 INTEL  DQ67SW    1072009 AMI     10013)
(XEN) ACPI: TCPA BC85DD30, 0032 (r2 INTEL  DQ67SW          1 MSFT  1000013)
(XEN) ACPI: SSDT BC85DD68, 0102 (r1 INTEL  DQ67SW          1 MSFT  3000001)
(XEN) ACPI: MCFG BC85DE70, 003C (r1 INTEL  DQ67SW    1072009 MSFT       97)
(XEN) ACPI: HPET BC85DEB0, 0038 (r1 INTEL  DQ67SW    1072009 AMI.        4)
(XEN) ACPI: ASF! BC85DEE8, 00A0 (r32 INTEL  DQ67SW          1 TFSM    F4240)
(XEN) ACPI: DMAR BC85DF88, 00E8 (r1 INTEL  DQ67SW          1 INTL        1)
(XEN) System RAM: 32683MB (33467896kB)
(XEN) Domain heap initialised
(XEN) ACPI: 32/64X FACS address mismatch in FADT - bcbdbf80/0000000000000000, using 32
(XEN) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) PCI: Not using MCFG for segment 0000 bus 00-3f
(XEN) Switched to APIC driver x2apic_cluster
(XEN) CPU0: 1600 ... 3100 MHz
(XEN) xstate: size: 0x340 and states: 0x7
(XEN) Speculative mitigation facilities:
(XEN)   Hardware hints:
(XEN)   Hardware features: IBPB IBRS STIBP SSBD L1D_FLUSH
(XEN)   Compiled-in support: INDIRECT_THUNK SHADOW_PAGING
(XEN)   Xen settings: BTI-Thunk RETPOLINE, SPEC_CTRL: IBRS- STIBP- SSBD-, Other: IBPB L1D_FLUSH BRANCH_HARDEN
(XEN)   L1TF: believed vulnerable, maxphysaddr L1D 46, CPUID 36, Safe address 1000000000
(XEN)   Support for HVM VMs: MSR_SPEC_CTRL RSB EAGER_FPU
(XEN)   Support for PV VMs: MSR_SPEC_CTRL RSB EAGER_FPU
(XEN)   XPTI (64-bit PV only): Dom0 enabled, DomU enabled (without PCID)
(XEN)   PV L1TF shadowing: Dom0 disabled, DomU disabled
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler
(XEN) Platform timer is 14.318MHz HPET
(XEN) Detected 3092.988 MHz processor.
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Posted Interrupt not enabled.
(XEN) Intel VT-d Shared EPT tables not enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) Allocated console ring of 16 KiB.
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) VMX: Disabling executable EPT superpages due to CVE-2018-12207
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB
(XEN) Brought up 4 CPUs
(XEN) Scheduling granularity: cpu, 1 CPU per sched-resource
(XEN) Dom0 has maximum 440 PIRQs
(XEN) Bogus DMIBAR 0xfed18001 on 0000:00:00.0
(XEN) WARNING: PVH is an experimental mode with limited functionality
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Scrubbing Free RAM in background
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
(XEN) Freed 600kB init memory
---<<BOOT>>---
Copyright (c) 1992-2021 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 13.1-STABLE #0 stable/13-984a45d77: Sat Jun  4 07:38:09 PDT 2022
    buhrow@fbsd_dev.nfbcal.org:/usr/home/buhrow/obj/usr/home/buhrow/src/fbsd-src/13/amd64.amd64/sys/GENERIC amd64
FreeBSD clang version 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)
VT(vga): text 80x25
XEN: Hypervisor version 4.16 detected.
CPU microcode: no matching update found
CPU: Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz (3092.99-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x206a7  Family=0x6  Model=0x2a  Stepping=7
  Features=0x1fc3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT>
  Features2=0x9fba2203<SSE3,PCLMULQDQ,SSSE3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,HV>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  Structured Extended Features3=0x9c000000<IBPB,STIBP,L1DFL,SSBD>
  XSAVE Features=0x1<XSAVEOPT>
  AMD Extended Feature Extensions ID EBX=0x101000<IBPB>
Hypervisor: Origin = "XenVMMXenVMM"
real memory  = 9725026304 (9274 MB)
avail memory = 8273047552 (7889 MB)
Event timer "LAPIC" quality 100
ACPI APIC Table: <INTEL  DQ67SW  >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
random: unblocking device.
ioapic0 <Version 1.1> irqs 0-23
Launching APs: 1
random: entropy device external interface
kbd1 at kbdmux0
vtvga0: <VT VGA driver>
smbios0: <System Management BIOS> at iomem 0xf0480-0xf049e
smbios0: Version: 2.6, BCD Revision: 2.6
aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS>
acpi0: <INTEL DQ67SW>
acpi0: Power Button (fixed)
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
hpet0: memory region width 1024 too small for 32 timers
device_attach: hpet0 attach returned 6
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
[ ... At this point, the system hangs, needing a hard reset ...]


<Successful boot of stable/13, without xen>
BIOS drive C: is disk0
BIOS drive D: is disk1
BIOS drive E: is disk2
BIOS drive F: is disk3
BIOS 614kB/523264kB available memory

FreeBSD/x86 bootstrap loader, Revision 1.1
Loading /boot/defaults/loader.conf
Loading /boot/defaults/loader.conf
Loading /boot/device.hints
Loading /boot/loader.conf
Loading /boot/loader.conf.local
OK  unset xen_kernel
OK boot
Loading kernel...
boot/kernel/kernel text=0x1855c8 text=0xe02330 /-\|/-\text=0x665524 |/-data=0x140 data=0x1c0448+0x43ebb8 \syms=[0x8+0x18a350|+0x8+0x1a8b68/]
Loading configured modules...
/etc/hostid size=0x25
/boot/kernel/tpm.ko size 0xad70 at 0x2133000
/boot/entropy size=0x1000
/boot/kernel/ipmi.ko /size 0x11950 at 0x213f000
loading required module 'smbus'
/boot/kernel/smbus.ko size 0x3cb0 at 0x2151000
/boot/firmware/intel-ucode.bin \|size=0x303800
/boot/kernel/geom_mirror.ko size 0x20c80 at 0x2459000
---<<BOOT>>---
Copyright (c) 1992-2021 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 13.1-STABLE #0 stable/13-984a45d77: Sat Jun  4 07:38:09 PDT 2022
    buhrow@fbsd_dev.nfbcal.org:/usr/home/buhrow/obj/usr/home/buhrow/src/fbsd-src/13/amd64.amd64/sys/GENERIC amd64
FreeBSD clang version 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)
VT(vga): resolution 640x480
CPU microcode: updated from 0x2e to 0x2f
CPU: Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz (3093.10-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x206a7  Family=0x6  Model=0x2a  Stepping=7
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x1fbae3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  Structured Extended Features3=0x9c000400<MD_CLEAR,IBPB,STIBP,L1DFL,SSBD>
  XSAVE Features=0x1<XSAVEOPT>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
  TSC: P-state invariant, performance statistics
real memory  = 34359738368 (32768 MB)
avail memory = 33272487936 (31731 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <INTEL  DQ67SW  >
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
random: unblocking device.
ioapic0 <Version 2.0> irqs 0-23
Launching APs: 2 3 1
random: entropy device external interface
kbd1 at kbdmux0
vtvga0: <VT VGA driver>
smbios0: <System Management BIOS> at iomem 0xf0480-0xf049e
smbios0: Version: 2.6, BCD Revision: 2.6
aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS>
acpi0: <INTEL DQ67SW>
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 550
Event timer "HPET1" frequency 14318180 Hz quality 440
Event timer "HPET2" frequency 14318180 Hz quality 440
Event timer "HPET3" frequency 14318180 Hz quality 440
Event timer "HPET4" frequency 14318180 Hz quality 440
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
mps0: <Avago Technologies (LSI) SAS2008> port 0xe000-0xe0ff mem 0xfe6c0000-0xfe6c3fff,0xfe680000-0xfe6bffff irq 16 at device 0.0 on pci1
mps0: Firmware: 15.00.00.00, Driver: 21.02.00.00-fbsd
mps0: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
vgapci0: <VGA-compatible display> port 0xf000-0xf03f mem 0xfe000000-0xfe3fffff,0xd0000000-0xdfffffff irq 16 at device 2.0 on pci0
vgapci0: Boot video device
pci0: <simple comms> at device 22.0 (no driver attached)
atapci0: <Intel ATA controller> port 0xf130-0xf137,0xf120-0xf123,0xf110-0xf117,0xf100-0xf103,0xf0f0-0xf0ff irq 18 at device 22.2 on pci0
ata2: <ATA channel> at channel 0 on atapci0
ata3: <ATA channel> at channel 1 on atapci0
uart2: <Intel AMT - KT Controller> port 0xf0e0-0xf0e7 mem 0xfe729000-0xfe729fff irq 17 at device 22.3 on pci0
uart2: Using 1 MSI message
em0: <Intel(R) 82579LM> port 0xf080-0xf09f mem 0xfe700000-0xfe71ffff,0xfe728000-0xfe728fff irq 20 at device 25.0 on pci0
em0: EEPROM V0.13-4
em0: Using 1024 TX descriptors and 1024 RX descriptors
em0: Using an MSI interrupt
em0: Ethernet address: 00:22:4d:56:3c:f7
em0: netmap queues/slots: TX 1/1024, RX 1/1024
ehci0: <Intel Cougar Point USB 2.0 controller> mem 0xfe727000-0xfe7273ff irq 16 at device 26.0 on pci0
usbus0: EHCI version 1.0
usbus0 on ehci0
usbus0: 480Mbps High Speed USB v2.0
hdac0: <Intel Cougar Point HDA Controller> mem 0xfe720000-0xfe723fff irq 22 at device 27.0 on pci0
pcib2: <ACPI PCI-PCI bridge> irq 17 at device 28.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> irq 18 at device 28.6 on pci0
pci3: <ACPI PCI bus> on pcib3
xhci0: <NEC uPD720200 USB 3.0 controller> mem 0xfe500000-0xfe501fff irq 18 at device 0.0 on pci3
xhci0: 32 bytes context size, 32-bit DMA
usbus1 on xhci0
usbus1: 5.0Gbps Super Speed USB v3.0
ehci1: <Intel Cougar Point USB 2.0 controller> mem 0xfe726000-0xfe7263ff irq 23 at device 29.0 on pci0
usbus2: EHCI version 1.0
usbus2 on ehci1
usbus2: 480Mbps High Speed USB v2.0
pcib4: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci4: <ACPI PCI bus> on pcib4
pci4: <serial bus, FireWire> at device 3.0 (no driver attached)
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci0: <Intel Cougar Point AHCI SATA controller> port 0xf0d0-0xf0d7,0xf0c0-0xf0c3,0xf0b0-0xf0b7,0xf0a0-0xf0a3,0xf060-0xf07f mem 0xfe725000-0xfe7257ff irq 19 at device 31.2 on pci0
ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahciem0: <AHCI enclosure management bridge> on ahci0
acpi_button0: <Power Button> on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0uart0: console (9600,n,8,1)
tpm0: <Trusted Platform Module> iomem 0xfed40000-0xfed44fff on acpi0
tpm: WEC WPCT200 rev 0x46
WARNING: Device "tpm" is Giant locked and may be deleted before FreeBSD 14.0.
orm0: <ISA Option ROM> at iomem 0xd4800-0xd57ff pnpid ORM0000 on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbdc0: non-PNP ISA device will be removed from GENERIC in FreeBSD 14.
est0: <Enhanced SpeedStep Frequency Control> on cpu0
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 278d00001f00
device_attach: est0 attach returned 6
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 278d00001f00
device_attach: est1 attach returned 6
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 278d00001f00
device_attach: est2 attach returned 6
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 278d00001f00
device_attach: est3 attach returned 6
Timecounter "TSC-low" frequency 1546486788 Hz quality 1000
Timecounters tick every 1.000 msec
hdacc0: <Realtek ALC888 HDA CODEC> at cad 2 on hdac0
hdaa0: <Realtek ALC888 Audio Function Group> at nid 1 on hdacc0
pcm0: <Realtek ALC888 (Rear Analog)> at nid 20 and 24,26 on hdaa0
pcm1: <Realtek ALC888 (Front Analog)> at nid 27 and 25 on hdaa0
pcm2: <Realtek ALC888 (Onboard Digital)> at nid 30 on hdaa0
pcm3: <Realtek ALC888 (Onboard Analog Speaker)> at nid 23 on hdaa0
hdacc1: <Intel Cougar Point HDA CODEC> at cad 3 on hdac0
hdaa1: <Intel Cougar Point Audio Function Group> at nid 1 on hdacc1
pcm4: <Intel Cougar Point (HDMI/DP 8ch)> at nid 6 on hdaa1
Trying to mount root from ufs:/dev/mirror/gptroot []...
ugen2.1: <Intel EHCI root HUB> at usbus2
ugen1.1: <(0x1033) XHCI root HUB> at usbus1
ugen0.1: <Intel EHCI root HUB> at usbus0
uhub0 on usbus2
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus2
uhub1 on usbus0
uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
uhub2 on usbus1
uhub2: <(0x1033) XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus1
uhub2: 4 ports with 4 removable, self powered
uhub0: 3 ports with 3 removable, self powered
Root mount waiting for: CAM usbus0 usbus2
uhub1: 3 ports with 3 removable, self powered
ugen2.2: <vendor 0x8087 product 0x0024> at usbus2
uhub3 on uhub0
uhub3: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus2
ses0 at ahciem0 bus 0 scbus4 target 0 lun 0
ses0: <AHCI SGPIO Enclosure 2.00 0001> SEMB S-E-S 2.00 device
ses0: SEMB SES Device
ses0: pass4,cd0 in 'Slot 00', SATA Slot: scbus3 target 0
cd0 at ahcich0 bus 0 scbus3 target 0 lun 0
cd0: <ATAPI iHAS224   B GL0B> Removable CD-ROM SCSI device
cd0: Serial Number 3524220 228217500534
cd0: 150.000MB/s transfers (SATA 1.x, UDMA5, ATAPI 12bytes, PIO 8192bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed
da0 at mps0 bus 0 scbus0 target 0 lun 0
da0: <ATA WDC WD81PURZ-85L 0A80> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number VAJ3Y51L            
da0: 600.000MB/s transfers
da0: Command Queueing enabled
da0: 7630885MB (15628053168 512 byte sectors)
da2 at mps0 bus 0 scbus0 target 2 lun 0
da2: <ATA WDC WD82PURZ-85T 0A82> Fixed Direct Access SPC-4 SCSI device
da2: Serial Number VGKUGHSG            
da2: 600.000MB/s transfers
da2: Command Queueing enabled
da2: 7630885MB (15628053168 512 byte sectors)
da3 at mps0 bus 0 scbus0 target 3 lun 0
da3: <ATA WDC WD81PURZ-85L 0A80> Fixed Direct Access SPC-4 SCSI device
da3: Serial Number VAJYL6HL            
da3: 600.000MB/s transfers
da3: Command Queueing enabled
da3: 7630885MB (15628053168 512 byte sectors)
ugen0.2: <vendor 0x8087 product 0x0024> at usbus0
da1 at mps0 bus 0 scbus0 target 1 lun 0
da1: <ATA WDC WD81PURZ-85L 0A80> Fixed Direct Access SPC-4 SCSI device
da1: Serial Number VAJ3RZJL            
da1: 600.000MB/s transfers
da1: Command Queueing enabled
da1: 7630885MB (15628053168 512 byte sectors)
uhub4 on uhub1
uhub4: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus0
GEOM_MIRROR: Device mirror/gptroot launched (1/2).
GEOM_MIRROR: Device gptroot: rebuilding provider da0p2.
GEOM_MIRROR: Device mirror/gptswap launched (2/2).
GEOM_MIRROR: Device mirror/gptback launched (2/2).
Root mount waiting for: usbus0 usbus2
uhub3: 8 ports with 8 removable, self powered
uhub4: 6 ports with 6 removable, self powered
sysctl: kern.timecounter.hardware=XENTIMER at line 13: Invalid argument
Setting hostuuid: 49434d53-0200-9009-2500-09902500dac8.
Setting hostid: 0x0535156f.
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
GEOM: zvol/xendisks/loth_9_install: invalid disklabel.
Starting file system checks:
/dev/mirror/gptroot: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/mirror/gptroot: clean, 56609677 free (27589 frags, 7072761 blocks, 0.0% fragmentation)
Mounting local filesystems:.
Setting hostname: xen-lothlorien.nfbcal.org.
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat/pkg /usr/local/lib/gcc10 /usr/local/lib/perl5/5.32/mach/CORE
32-bit compatibility ldconfig path: /usr/lib32
Setting up harvesting: [UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED
Feeding entropy: .
Autoloading module: ichsmb
ichsmb0: <Intel Cougar Point SMBus controller> port 0xf040-0xf05f mem 0xfe724000-0xfe7240ff irq 18 at device 31.3 on pci0
smbus0: <System Management Bus> on ichsmb0
bridge0: Ethernet address: 58:9c:fc:10:94:73
Created clone inlo0: link state changed to UP
terfaces: bridge0.
em0: promiscuous mode enabled
bridge0: link state changed to UP
Starting Network: lo0 em0 bridge0.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
	inet 127.0.0.1 netmask 0xff0000em0: link state changed to UP
em0.22: link state changed to UP
00
	groups: lo
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
em0: flags=8963<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=4812098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,NOMAP>
	ether 00:22:4d:56:3c:f7
	media: Ethernet autoselect
	status: no carrier
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	ether 58:9c:fc:10:94:73
	id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
	maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
	member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
	        ifmaxaddr 0 port 1 priority 128 path cost 2000000
	groups: bridge
	nd6 options=9<PERFORMNUD,IFDISABLED>
Starting devd.
Sleeping 60  seconds before starting...
Waiting 60s for the default route interface: .....(no carrier)
add host 127.0.0.1: gateway lo0 fib 0: route already in table
add net default: gateway 10.40.60.1
add host ::1: gateway lo0 fib 0: route already in table
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
Updating /var/run/os-release done.
Creating and/or trimming log files.
Clearing /tmp (X related).
iUpdating motd:.