Re: panic: Unaligned free (was: kernel panic while copying files)
Date: Wed, 30 Jun 2021 12:57:03 UTC
On Wed, 30 Jun 2021 06:02:59 +0100
Graham Perrin <grahamperrin@gmail.com> wrote:
> On 29/06/2021 10:42, Gary Jennejohn wrote:
> > ___ panic is now the result of an unaligned free.
> >
> > panic: Unaligned free of 0xfffff800259e2800 from zone
> > 0xfffffe00dc9d2000(da_ccb) slab 0xfffff800259e2fd8(3)
> >
> > I have the crash dump and a debug kernel in case anyone wants more info.
> Can you post the backtrace etc. here? Thanks
>
Sure. As can be seen from the uma zone being da_ccb, the panic
resulted from setting kern.cam.da.enable_uma_ccbs=1.
Unread portion of the kernel message buffer:
panic: Unaligned free of 0xfffff800259e2800 from zone 0xfffffe00dc9d2000(da_ccb) slab 0xfffff800259e2fd8(3)
cpuid = 2
time = 1624958650
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2c/frame 0xfffffe00c62687a0
kdb_backtrace() at kdb_backtrace+0x46/frame 0xfffffe00c6268850
vpanic() at vpanic+0x227/frame 0xfffffe00c62688f0
panic() at panic+0x4e/frame 0xfffffe00c6268950
uma_dbg_free() at uma_dbg_free+0xfc/frame 0xfffffe00c62689a0
item_dtor() at item_dtor+0x7c/frame 0xfffffe00c62689e0
uma_zfree_arg() at uma_zfree_arg+0xf0/frame 0xfffffe00c6268a50
uma_zfree() at uma_zfree+0x23/frame 0xfffffe00c6268a70
xpt_free_ccb() at xpt_free_ccb+0x43/frame 0xfffffe00c6268a90
camperiphdone() at camperiphdone+0x211/frame 0xfffffe00c6268ae0
xpt_done_process() at xpt_done_process+0x550/frame 0xfffffe00c6268b40
xpt_done_td() at xpt_done_td+0x1c0/frame 0xfffffe00c6268b80
fork_exit() at fork_exit+0x117/frame 0xfffffe00c6268bf0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00c6268bf0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
KDB: enter: panic
doadump (textdump=0) at /usr/src/sys/kern/kern_shutdown.c:399
399 dumptid = curthread->td_tid;
(kgdb) bt
#0 doadump (textdump=0) at /usr/src/sys/kern/kern_shutdown.c:399
#1 0xffffffff804d5dd7 in db_dump (dummy=-2138843371, dummy2=false, dummy3=-1,
dummy4=0xfffffe00c6268320 "") at /usr/src/sys/ddb/db_command.c:575
#2 0xffffffff804d5bf4 in db_command (
last_cmdp=0xffffffff8114ce80 <db_last_command>, cmd_table=0x0, dopager=1)
at /usr/src/sys/ddb/db_command.c:482
#3 0xffffffff804d583c in db_command_loop ()
at /usr/src/sys/ddb/db_command.c:535
#4 0xffffffff804da27c in db_trap (type=3, code=0)
at /usr/src/sys/ddb/db_main.c:270
#5 0xffffffff8083df9d in kdb_trap (type=3, code=0, tf=0xfffffe00c6268770)
at /usr/src/sys/kern/subr_kdb.c:727
#6 0xffffffff80d31494 in trap (frame=0xfffffe00c6268770)
at /usr/src/sys/amd64/amd64/trap.c:604
#7 0xffffffff80d32628 in trap_check (frame=0xfffffe00c6268770)
at /usr/src/sys/amd64/amd64/trap.c:664
#8 <signal handler called>
#9 breakpoint () at /usr/src/sys/amd64/include/cpufunc.h:66
#10 0xffffffff8083d3d0 in kdb_enter (why=0xffffffff80e0355b "panic",
msg=0xffffffff80e0355b "panic") at /usr/src/sys/kern/subr_kdb.c:505
#11 0xffffffff807d1725 in vpanic (
fmt=0xffffffff80dbca46 "Unaligned free of %p from zone %p(%s) slab %p(%d)", ap=0xfffffe00c6268930) at /usr/src/sys/kern/kern_shutdown.c:906
#12 0xffffffff807d120e in panic (
fmt=0xffffffff80dbca46 "Unaligned free of %p from zone %p(%s) slab %p(%d)")
at /usr/src/sys/kern/kern_shutdown.c:843
#13 0xffffffff80c16a8c in uma_dbg_free (zone=0xfffffe00dc9d2000,
slab=0xfffff800259e2fd8, item=0xfffff800259e2800)
at /usr/src/sys/vm/uma_core.c:5659
#14 0xffffffff80c0c5dc in item_dtor (zone=0xfffffe00dc9d2000,
item=0xfffff800259e2800, size=544, udata=0x0, skip=SKIP_NONE)
at /usr/src/sys/vm/uma_core.c:3418
#15 0xffffffff80c0ba60 in uma_zfree_arg (zone=0xfffffe00dc9d2000,
item=0xfffff800259e2800, udata=0x0) at /usr/src/sys/vm/uma_core.c:4374
#16 0xffffffff802e45d3 in uma_zfree (zone=0xfffffe00dc9d2000,
item=0xfffff800259e2800) at /usr/src/sys/vm/uma.h:404
#17 0xffffffff802dc3c3 in xpt_free_ccb (free_ccb=0xfffff800259e2800)
at /usr/src/sys/cam/cam_xpt.c:4676
#18 0xffffffff802dacf1 in camperiphdone (periph=0xfffff80025329b00,
done_ccb=0xfffff80025a24cc0) at /usr/src/sys/cam/cam_periph.c:1427
#19 0xffffffff802e4520 in xpt_done_process (ccb_h=0xfffff80025a24cc0)
at /usr/src/sys/cam/cam_xpt.c:5493
#20 0xffffffff802e68e0 in xpt_done_td (arg=0xffffffff81143700 <cam_doneqs>)
at /usr/src/sys/cam/cam_xpt.c:5548
#21 0xffffffff807673c7 in fork_exit (callout=0xffffffff802e6720 <xpt_done_td>,
arg=0xffffffff81143700 <cam_doneqs>, frame=0xfffffe00c6268c00)
at /usr/src/sys/kern/kern_fork.c:1083
#22 <signal handler called>
[kgdb stuff removed]
(kgdb) down
#15 0xffffffff80c0ba60 in uma_zfree_arg (zone=0xfffffe00dc9d2000,
item=0xfffff800259e2800, udata=0x0) at /usr/src/sys/vm/uma_core.c:4374
4374 item_dtor(zone, item, cache_uz_size(cache), udata, SKIP_NONE);
(kgdb) down
#14 0xffffffff80c0c5dc in item_dtor (zone=0xfffffe00dc9d2000,
item=0xfffff800259e2800, size=544, udata=0x0, skip=SKIP_NONE)
at /usr/src/sys/vm/uma_core.c:3418
3418 uma_dbg_free(zone, NULL, item);
(kgdb) p/x skipdbg
$26 = 0x0
(kgdb) p/x zone->uz_flags
$27 = 0x41000000 (UMA_ZFLAG_TRASH|UMA_ZFLAG_CTORDTOR)
(kgdb) down
#13 0xffffffff80c16a8c in uma_dbg_free (zone=0xfffffe00dc9d2000,
slab=0xfffff800259e2fd8, item=0xfffff800259e2800)
at /usr/src/sys/vm/uma_core.c:5659
5659 panic("Unaligned free of %p from zone %p(%s) slab %p(%d)",
Note that item is the same as saved_ccb.
(kgdb) p/x *zone->uz_keg
$28 = {uk_hash = {uh_slab_hash = 0x0, uh_hashsize = 0x0, uh_hashmask = 0x0},
uk_zones = {lh_first = 0xfffffe00dc9d2000}, uk_dr = {
dr_policy = 0xffffffff810010e0, dr_iter = 0x0}, uk_align = 0x7,
uk_reserve = 0x0, uk_size = 0x220, uk_rsize = 0x220,
uk_init = 0xffffffff80c17d50, uk_fini = 0xffffffff80c17d80,
uk_allocf = 0xffffffff80d342f0, uk_freef = 0xffffffff80d346a0,
uk_offset = 0x0, uk_kva = 0x0, uk_pgoff = 0xfd8, uk_ppera = 0x1,
uk_ipers = 0x7, uk_flags = 0x0, uk_name = 0xffffffff80debbac, uk_link = {
le_next = 0xfffff80005968000, le_prev = 0xfffff80005968380},
uk_domain = 0xfffff80005968240}
(kgdb) p/x *slab
$29 = {us_link = {le_next = 0xdeadc0dedeadc0de, le_prev = 0xdeadc0dedeadc0de},
us_freecount = 0xc0de, us_flags = 0xad, us_domain = 0xde, us_free = {
__bits = 0xfffff800259e2ff0}}
(kgdb) p/x *0xfffff800259e2ff0
$30 = 0xdeadc0de
Don't know whether this matters, but slab seems to be unitialized.
(kgdb) p/x freei
$31 = 0x3
In any case, saved_ccb has an address which lies outside the range
covered by slab, i.e. freei is bigger than the number of entries in
slab.
I suspect that the only way to really figure out what's going on is to
run the kernel in kgbd and set lots of breakpoints,
--
Gary Jennejohn