Kernel crashes since FreeBSD 13.1-RC6

From: thilo jeremias <thilo_at_nispuk.com>
Date: Sat, 14 May 2022 13:08:46 UTC
Hello,

I have been noticing crashes since updating to 13.1-RC6
( I don’t rule out a HW problem, but it really just happens since the update)


They all fault somewhere in the ethernet areas (I’m having an Intel I350 and an unconnected on board Realtek)
( below:  fault messages, kernel version & partial system-config)

If someone knows, it’s my problem. Let me know. If I shall file a bug, ditto. If you need more info, ditto
On the next crash, I will downgrade to RC5 to rule out HW.

Thank you for all the work you put into this operating system!

Cheers Thilo

 perl -ne 'print if /Fatal trap/../BOOT/;' /var/log/messages 
May 12 20:20:50 maggi kernel: [475467] Fatal trap 12: page fault while in kernel mode
May 12 20:20:50 maggi kernel: [475467] cpuid = 1; apic id = 11
May 12 20:20:50 maggi kernel: [475467] fault virtual address    = 0x8
May 12 20:20:50 maggi kernel: [475467] fault code               = supervisor read data, page not present
May 12 20:20:50 maggi kernel: [475467] instruction pointer      = 0x20:0xffffffff80cb8048
May 12 20:20:50 maggi kernel: [475467] stack pointer            = 0x28:0xfffffe001bd35710
May 12 20:20:50 maggi kernel: [475467] frame pointer            = 0x28:0xfffffe001bd35750
May 12 20:20:50 maggi kernel: [475467] code segment             = base 0x0, limit 0xfffff, type 0x1b
May 12 20:20:50 maggi kernel: [475467]                  = DPL 0, pres 1, long 1, def32 0, gran 1
May 12 20:20:50 maggi kernel: [475467] processor eflags = interrupt enabled, resume, IOPL = 0
May 12 20:20:50 maggi kernel: [475467] current process          = 0 (if_io_tqg_1)
May 12 20:20:50 maggi kernel: [475467] trap number              = 12
May 12 20:20:50 maggi kernel: [475467] panic: page fault
May 12 20:20:50 maggi kernel: [475467] cpuid = 1
May 12 20:20:50 maggi kernel: [475467] time = 1652379494
May 12 20:20:50 maggi kernel: [475467] KDB: stack backtrace:
May 12 20:20:50 maggi kernel: [475467] #0 0xffffffff80c69465 at kdb_backtrace+0x65
May 12 20:20:50 maggi kernel: [475467] #1 0xffffffff80c1bb1f at vpanic+0x17f
May 12 20:20:50 maggi kernel: [475467] #2 0xffffffff80c1b993 at panic+0x43
May 12 20:20:50 maggi kernel: [475467] #3 0xffffffff810afdf5 at trap_fatal+0x385
May 12 20:20:50 maggi kernel: [475467] #4 0xffffffff810afe4f at trap_pfault+0x4f
May 12 20:20:50 maggi kernel: [475467] #5 0xffffffff81087528 at calltrap+0x8
May 12 20:20:50 maggi kernel: [475467] #6 0xffffffff80cb8257 at sbdrop+0x37
May 12 20:20:50 maggi kernel: [475467] #7 0xffffffff80dd7fab at tcp_do_segment+0x2dbb
May 12 20:20:50 maggi kernel: [475467] #8 0xffffffff80dd44b1 at tcp_input_with_port+0xb61
May 12 20:20:50 maggi kernel: [475467] #9 0xffffffff80dd515b at tcp_input+0xb
May 12 20:20:50 maggi kernel: [475467] #10 0xffffffff80dc691f at ip_input+0x11f
May 12 20:20:50 maggi kernel: [475467] #11 0xffffffff80d53089 at netisr_dispatch_src+0xb9
May 12 20:20:50 maggi kernel: [475467] #12 0xffffffff80d36ea8 at ether_demux+0x138
May 12 20:20:50 maggi kernel: [475467] #13 0xffffffff80d38235 at ether_nh_input+0x355
May 12 20:20:50 maggi kernel: [475467] #14 0xffffffff80d53089 at netisr_dispatch_src+0xb9
May 12 20:20:50 maggi kernel: [475467] #15 0xffffffff80d372d9 at ether_input+0x69
May 12 20:20:50 maggi kernel: [475467] #16 0xffffffff80d36e91 at ether_demux+0x121
May 12 20:20:50 maggi kernel: [475467] #17 0xffffffff80d38235 at ether_nh_input+0x355
May 12 20:20:50 maggi kernel: [475467] Uptime: 5d12h4m27s
May 12 20:20:50 maggi kernel: ---<<BOOT>>---
May 13 22:37:37 maggi kernel: [94562] Fatal trap 12: page fault while in kernel mode
May 13 22:37:37 maggi kernel: [94562] cpuid = 4; apic id = 14
May 13 22:37:37 maggi kernel: [94562] fault virtual address     = 0x8
May 13 22:37:37 maggi kernel: [94562] fault code                = supervisor read data, page not present
May 13 22:37:37 maggi kernel: [94562] instruction pointer       = 0x20:0xffffffff80cb8048
May 13 22:37:37 maggi kernel: [94562] stack pointer             = 0x28:0xfffffe001bd4e710
May 13 22:37:37 maggi kernel: [94562] frame pointer             = 0x28:0xfffffe001bd4e750
May 13 22:37:37 maggi kernel: [94562] code segment              = base 0x0, limit 0xfffff, type 0x1b
May 13 22:37:37 maggi kernel: [94562]                   = DPL 0, pres 1, long 1, def32 0, gran 1
May 13 22:37:37 maggi kernel: [94562] processor eflags  = interrupt enabled, resume, IOPL = 0
May 13 22:37:37 maggi kernel: [94562] current process           = 0 (if_io_tqg_4)
May 13 22:37:37 maggi kernel: [94562] trap number               = 12
May 13 22:37:37 maggi kernel: [94562] panic: page fault
May 13 22:37:37 maggi kernel: [94562] cpuid = 4
May 13 22:37:37 maggi kernel: [94562] time = 1652474125
May 13 22:37:37 maggi kernel: [94562] KDB: stack backtrace:
May 13 22:37:37 maggi kernel: [94562] #0 0xffffffff80c69465 at kdb_backtrace+0x65
May 13 22:37:37 maggi kernel: [94562] #1 0xffffffff80c1bb1f at vpanic+0x17f
May 13 22:37:37 maggi kernel: [94562] #2 0xffffffff80c1b993 at panic+0x43
May 13 22:37:37 maggi kernel: [94562] #3 0xffffffff810afdf5 at trap_fatal+0x385
May 13 22:37:37 maggi kernel: [94562] #4 0xffffffff810afe4f at trap_pfault+0x4f
May 13 22:37:37 maggi kernel: [94562] #5 0xffffffff81087528 at calltrap+0x8
May 13 22:37:37 maggi kernel: [94562] #6 0xffffffff80cb8257 at sbdrop+0x37
May 13 22:37:37 maggi kernel: [94562] #7 0xffffffff80dd7fab at tcp_do_segment+0x2dbb
May 13 22:37:37 maggi kernel: [94562] #8 0xffffffff80dd44b1 at tcp_input_with_port+0xb61
May 13 22:37:37 maggi kernel: [94562] #9 0xffffffff80dd515b at tcp_input+0xb
May 13 22:37:37 maggi kernel: [94562] #10 0xffffffff80dc691f at ip_input+0x11f
May 13 22:37:37 maggi kernel: [94562] #11 0xffffffff80d53089 at netisr_dispatch_src+0xb9
May 13 22:37:37 maggi kernel: [94562] #12 0xffffffff80d36ea8 at ether_demux+0x138
May 13 22:37:37 maggi kernel: [94562] #13 0xffffffff80d38235 at ether_nh_input+0x355
May 13 22:37:37 maggi kernel: [94562] #14 0xffffffff80d53089 at netisr_dispatch_src+0xb9
May 13 22:37:37 maggi kernel: [94562] #15 0xffffffff80d372d9 at ether_input+0x69
May 13 22:37:37 maggi kernel: [94562] #16 0xffffffff80d36e91 at ether_demux+0x121
May 13 22:37:37 maggi kernel: [94562] #17 0xffffffff80d38235 at ether_nh_input+0x355
May 13 22:37:37 maggi kernel: [94562] Uptime: 1d2h16m2s
May 13 22:37:37 maggi kernel: ---<<BOOT>>---
May 14 14:44:43 maggi kernel: [57974] Fatal trap 12: page fault while in kernel mode
May 14 14:44:43 maggi kernel: [57974] cpuid = 5; apic id = 15
May 14 14:44:43 maggi kernel: [57974] fault virtual address     = 0x8
May 14 14:44:43 maggi kernel: [57974] fault code                = supervisor read data, page not present
May 14 14:44:43 maggi kernel: [57974] instruction pointer       = 0x20:0xffffffff80cb8048
May 14 14:44:43 maggi kernel: [57974] stack pointer             = 0x28:0xfffffe001bd49710
May 14 14:44:43 maggi kernel: [57974] frame pointer             = 0x28:0xfffffe001bd49750
May 14 14:44:43 maggi kernel: [57974] code segment              = base 0x0, limit 0xfffff, type 0x1b
May 14 14:44:43 maggi kernel: [57974]                   = DPL 0, pres 1, long 1, def32 0, gran 1
May 14 14:44:43 maggi kernel: [57974] processor eflags  = interrupt enabled, resume, IOPL = 0
May 14 14:44:43 maggi kernel: [57974] current process           = 0 (if_io_tqg_5)
May 14 14:44:43 maggi kernel: [57974] trap number               = 12
May 14 14:44:43 maggi kernel: [57974] panic: page fault
May 14 14:44:43 maggi kernel: [57974] cpuid = 5
May 14 14:44:43 maggi kernel: [57974] time = 1652532164
May 14 14:44:43 maggi kernel: [57974] KDB: stack backtrace:
May 14 14:44:43 maggi kernel: [57974] #0 0xffffffff80c69465 at kdb_backtrace+0x65
May 14 14:44:43 maggi kernel: [57974] #1 0xffffffff80c1bb1f at vpanic+0x17f
May 14 14:44:43 maggi kernel: [57974] #2 0xffffffff80c1b993 at panic+0x43
May 14 14:44:43 maggi kernel: [57974] #3 0xffffffff810afdf5 at trap_fatal+0x385
May 14 14:44:43 maggi kernel: [57974] #4 0xffffffff810afe4f at trap_pfault+0x4f
May 14 14:44:43 maggi kernel: [57974] #5 0xffffffff81087528 at calltrap+0x8
May 14 14:44:43 maggi kernel: [57974] #6 0xffffffff80cb8257 at sbdrop+0x37
May 14 14:44:43 maggi kernel: [57974] #7 0xffffffff80dd7fab at tcp_do_segment+0x2dbb
May 14 14:44:43 maggi kernel: [57974] #8 0xffffffff80dd44b1 at tcp_input_with_port+0xb61
May 14 14:44:43 maggi kernel: [57974] #9 0xffffffff80dd515b at tcp_input+0xb
May 14 14:44:43 maggi kernel: [57974] #10 0xffffffff80dc691f at ip_input+0x11f
May 14 14:44:43 maggi kernel: [57974] #11 0xffffffff80d53089 at netisr_dispatch_src+0xb9
May 14 14:44:43 maggi kernel: [57974] #12 0xffffffff80d36ea8 at ether_demux+0x138
May 14 14:44:43 maggi kernel: [57974] #13 0xffffffff80d38235 at ether_nh_input+0x355
May 14 14:44:43 maggi kernel: [57974] #14 0xffffffff80d53089 at netisr_dispatch_src+0xb9
May 14 14:44:43 maggi kernel: [57974] #15 0xffffffff80d372d9 at ether_input+0x69
May 14 14:44:43 maggi kernel: [57974] #16 0xffffffff80d36e91 at ether_demux+0x121
May 14 14:44:43 maggi kernel: [57974] #17 0xffffffff80d38235 at ether_nh_input+0x355
May 14 14:44:43 maggi kernel: [57974] Uptime: 16h6m14s
May 14 14:44:43 maggi kernel: ---<<BOOT>>---

uname -a
FreeBSD maggi.nispuk.com 13.1-RC6 FreeBSD 13.1-RC6 releng/13.1-n250146-cd651fcc491 GENERIC amd64

freebsd-version -kru
13.1-RC6
13.1-RC6
13.1-RC6

dmesg (truncated)

Copyright (c) 1992-2021 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 13.1-RC6 releng/13.1-n250146-cd651fcc491 GENERIC amd64
FreeBSD clang version 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)
VT(vga): resolution 640x480
module_register: cannot register pci/em from kernel; already loaded from if_igb.ko
Module pci/em failed to register: 17
module_register: cannot register pci/igb from kernel; already loaded from if_igb.ko
Module pci/igb failed to register: 17
CPU: AMD FX-8320E Eight-Core Processor               (3193.18-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x600f20  Family=0x15  Model=0x2  Stepping=0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x3e98320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0x1ebbfff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,XOP,SKINIT,WDT,LWP,FMA4,TCE,NodeId,TBM,Topology,PCXC,PNXC>
  Structured Extended Features=0x8<BMI1>
  SVM: NP,NRIP,VClean,AFlush,DAssist,NAsids=65536
  TSC: P-state invariant, performance statistics
real memory  = 17179869184 (16384 MB)
avail memory = 16570400768 (15802 MB)
Event timer "LAPIC" quality 100
ACPI APIC Table: <ALASKA A M I>
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 8 core(s)
random: unblocking device.
Firmware Warning (ACPI): Optional FADT field Pm2ControlBlock has valid Length but zero Address: 0x0000000000000000/0x1 (20201113/tbfadt-796)
ioapic0 <Version 2.1> irqs 0-23
ioapic1 <Version 2.1> irqs 24-55
Launching APs: 1 3 4 5 6 7 2
random: entropy device external interface
kbd1 at kbdmux0
vtvga0: <VT VGA driver>
smbios0: <System Management BIOS> at iomem 0xf04c0-0xf04de
smbios0: Version: 2.7, BCD Revision: 2.7
aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS>
acpi0: <ALASKA A M I>
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pci0: <base peripheral, IOMMU> at device 0.2 (no driver attached)
pcib1: <ACPI PCI-PCI bridge> irq 52 at device 2.0 on pci0
pci1: <ACPI PCI bus> on pcib1
vgapci0: <VGA-compatible display> port 0xe000-0xe0ff mem 0xc0000000-0xcfffffff,0xfea20000-0xfea3ffff irq 24 at device 0.0 on pci1
vgapci0: Boot video device
hdac0: <ATI RV910 HDA Controller> mem 0xfea40000-0xfea43fff irq 25 at device 0.1 on pci1
pcib2: <ACPI PCI-PCI bridge> irq 52 at device 4.0 on pci0
pci2: <ACPI PCI bus> on pcib2
igb0: <Intel(R) I350 (Copper)> mem 0xfe400000-0xfe4fffff,0xfe70c000-0xfe70ffff irq 44 at device 0.0 on pci2
igb0: EEPROM V1.67-0 Option ROM V19-b5-p12 eTrack 0x80000fb7
igb0: Using 1024 TX descriptors and 1024 RX descriptors
igb0: Using 8 RX queues 8 TX queues
igb0: Using MSI-X interrupts with 9 vectors
igb0: Ethernet address: a0:36:9f:88:18:58
igb0: netmap queues/slots: TX 8/1024, RX 8/1024
igb1: <Intel(R) I350 (Copper)> mem 0xfe300000-0xfe3fffff,0xfe708000-0xfe70bfff irq 45 at device 0.1 on pci2
igb1: EEPROM V1.67-0 Option ROM V19-b5-p12 eTrack 0x80000fb7
igb1: Using 1024 TX descriptors and 1024 RX descriptors
igb1: Using 8 RX queues 8 TX queues
igb1: Using MSI-X interrupts with 9 vectors
igb1: Ethernet address: a0:36:9f:88:18:59
igb1: netmap queues/slots: TX 8/1024, RX 8/1024
igb2: <Intel(R) I350 (Copper)> mem 0xfe200000-0xfe2fffff,0xfe704000-0xfe707fff irq 46 at device 0.2 on pci2
igb2: EEPROM V1.67-0 Option ROM V19-b5-p12 eTrack 0x80000fb7
igb2: Using 1024 TX descriptors and 1024 RX descriptors
igb2: Using 8 RX queues 8 TX queues
igb2: Using MSI-X interrupts with 9 vectors
igb2: Ethernet address: a0:36:9f:88:18:5a
igb2: netmap queues/slots: TX 8/1024, RX 8/1024
igb3: <Intel(R) I350 (Copper)> mem 0xfe100000-0xfe1fffff,0xfe700000-0xfe703fff irq 47 at device 0.3 on pci2
igb3: EEPROM V1.67-0 Option ROM V19-b5-p12 eTrack 0x80000fb7
igb3: Using 1024 TX descriptors and 1024 RX descriptors
igb3: Using 8 RX queues 8 TX queues
igb3: Using MSI-X interrupts with 9 vectors
igb3: Ethernet address: a0:36:9f:88:18:5b
igb3: netmap queues/slots: TX 8/1024, RX 8/1024
pcib3: <ACPI PCI-PCI bridge> irq 53 at device 9.0 on pci0
pci3: <ACPI PCI bus> on pcib3
xhci0: <XHCI (generic) USB 3.0 controller> mem 0xfe900000-0xfe907fff irq 48 at device 0.0 on pci3
xhci0: 64 bytes context size, 64-bit DMA
usbus0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
ahci0: <AMD SB7x0/SB8x0/SB9x0 AHCI SATA controller> port 0xf040-0xf047,0xf030-0xf033,0xf020-0xf027,0xf010-0xf013,0xf000-0xf00f mem 0xfeb0b000-0xfeb0b3ff irq 19 at device 17.0 on pci0