panic: general protection fault -- ZFS scrub/checkum related
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Mon, 24 Mar 2025 10:53:47 UTC
On stock FreeBSD 13.4-RELEASE-p1 GENERIC amd64 -- after a dead disk replacement in a raidz3-0 pool; after a successful re-silvering; subsequent scrubs show one of the non-replaced disks racking up checksum errors quite rapidly - and (each time sofar, not caught it in the actual act yet) at around 30% and around 299 checksum errors; crash with below kernel panic. Does this ring a bell with anyone ? Dw # tar zxf kernel-dbg.txz -C / # kgdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.0 ... substantially indentical to below # kgdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.1 ... substantially indentical to below # kgdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.2 ... substantially indentical to below # kgdb /usr/lib/debug/boot/kernel/kernel.debug vmcore.3 GNU gdb (GDB) 15.1 [GDB v15.1 for FreeBSD] .. Unread portion of the kernel message buffer: .... <7>sonewconn: pcb 0xfffff800623027c0 (148.251.234.232:993 (proto 6)): Listen queue overflow: 49 already in queue awaiting acceptance (4 occurrences) <118>2025-03-24T10:58:12.552431+01:00 weser.webweaving.org bandwidthd 1338 - - IP_NUM is too low, dropping ip.... Fatal trap 9: general protection fault while in kernel mode cpuid = 0; apic id = 00 instruction pointer = 0x20:0xffffffff80cf2535 stack pointer = 0x28:0xfffffe0261056e88 frame pointer = 0x28:0xfffffe0261056eb0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 43896 (ps) trap number = 9 panic: general protection fault cpuid = 0 time = 1742810293 KDB: stack backtrace: #0 0xffffffff80c3dd05 at kdb_backtrace+0x65 #1 0xffffffff80bf1882 at vpanic+0x152 #2 0xffffffff80bf1723 at panic+0x43 #3 0xffffffff810ae069 at trap_fatal+0x389 #4 0xffffffff81085098 at calltrap+0x8 #5 0xffffffff80bd5e9c at fill_kinfo_proc+0x72c #6 0xffffffff80bd6484 at kern_proc_out+0x34 #7 0xffffffff80bd8d46 at sysctl_out_proc+0x66 #8 0xffffffff80bd6d21 at proc_iterate+0x111 #9 0xffffffff80bd8cb2 at sysctl_kern_proc+0xf2 #10 0xffffffff80c022d1 at sysctl_root_handler_locked+0x91 #11 0xffffffff80c016e4 at sysctl_root+0x244 #12 0xffffffff80c01d5e at userland_sysctl+0x15e #13 0xffffffff80c01bc0 at sys___sysctl+0x60 #14 0xffffffff810ae933 at amd64_syscall+0x103 #15 0xffffffff810859ab at fast_syscall_common+0xf8 Uptime: 14h20m32s Dumping 9139 out of 32415 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:53 warning: 53 /usr/src/sys/amd64/include/pcpu_aux.h: No such file or directory (kgdb) bt #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:53 #1 doadump (textdump=<optimized out>) at /usr/src/sys/kern/kern_shutdown.c:394 #2 0xffffffff80bf144e in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:482 #3 0xffffffff80bf18ef in vpanic (fmt=0xffffffff8120c09e "%s", ap=ap@entry=0xfffffe0261056d30) at /usr/src/sys/kern/kern_shutdown.c:921 #4 0xffffffff80bf1723 in panic (fmt=<unavailable>) at /usr/src/sys/kern/kern_shutdown.c:845 #5 0xffffffff810ae069 in trap_fatal (frame=0xfffffe0261056dc0, eva=0) at /usr/src/sys/amd64/amd64/trap.c:940 #6 <signal handler called> #7 strlcpy (dst=<optimized out>, src=<optimized out>, dsize=dsize@entry=9) at /usr/src/sys/libkern/strlcpy.c:37 #8 0xffffffff80bd6024 in fill_kinfo_thread (td=0xfffff8066b02c740, kp=kp@entry=0xfffffe0261057220, preferthread=preferthread@entry=0) at /usr/src/sys/kern/kern_proc.c:1262 #9 0xffffffff80bd5e9c in fill_kinfo_proc (p=p@entry=0xfffffe0143907570, kp=kp@entry=0xfffffe0261057220) at /usr/src/sys/kern/kern_proc.c:1368 #10 0xffffffff80bd6484 in kern_proc_out (p=0xfffffe02610573bb, p@entry=0xfffffe0143907570, sb=sb@entry=0xfffffe02610576a0, flags=8, flags@entry=1) at /usr/src/sys/kern/kern_proc.c:1551 #11 0xffffffff80bd8d46 in sysctl_out_proc (p=0xfffffe0143907570, req=0xfffffe0261057cc8, flags=1) at /usr/src/sys/kern/kern_proc.c:1594 #12 0xffffffff80bd6d21 in proc_iterate (cb=0xffffffff80bd8e30 <sysctl_kern_proc_iterate>, cbarg=cbarg@entry=0xfffffe0261057ba8) at /usr/src/sys/kern/kern_proc.c:1617 #13 0xffffffff80bd8cb2 in sysctl_kern_proc (oidp=<optimized out>, arg1=0xfffffe0261057d8c, arg2=<optimized out>, req=0xfffffe0261057cc8) at /usr/src/sys/kern/kern_proc.c:1790 #14 0xffffffff80c022d1 in sysctl_root_handler_locked (oid=oid@entry=0xffffffff81ccd0a0 <sysctl___kern_proc_proc>, arg1=arg1@entry=0xfffffe0261057d8c, arg2=arg2@entry=0, req=0xfffffe0261057cc8, tracker=tracker@entry=0xfffffe0261057c48) at /usr/src/sys/kern/kern_sysctl.c:183 #15 0xffffffff80c016e4 in sysctl_root (oidp=<optimized out>, arg1=0xfffffe0261057d8c, arg1@entry=0xfffffe0261057d80, arg2=0, arg2@entry=3, req=req@entry=0xfffffe0261057cc8) at /usr/src/sys/kern/kern_sysctl.c:2302 #16 0xffffffff80c01d5e in userland_sysctl (td=td@entry=0xfffff80433510740, name=name@entry=0xfffffe0261057d80, namelen=<optimized out>, old=<optimized out>, oldlenp=<optimized out>, inkernel=inkernel@entry=0, new=0x0, newlen=0, retval=0xfffffe0261057de8, flags=0) at /usr/src/sys/kern/kern_sysctl.c:2459 #17 0xffffffff80c01bc0 in sys___sysctl (td=0xfffff80433510740, uap=0xfffff80433510b28) at /usr/src/sys/kern/kern_sysctl.c:2332 #18 0xffffffff810ae933 in syscallenter (td=0xfffff80433510740) at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:188 #19 amd64_syscall (td=0xfffff80433510740, traced=0) at /usr/src/sys/amd64/amd64/trap.c:1181 #20 <signal handler called> #21 0x0000274db155238a in ?? () Backtrace stopped: Cannot access memory at address 0x274dadb15168 (kgdb) quit