devfs panic w/INVARIANTS

[Andrew Gallatin]

I've got a commercial driver that uses device cloning.
At unload time, the driver calls clone_cleanup(). When I unload
the driver when the kernel is built with INVARIANTS, I'll see a
panic in devfs_populate_loop().  This happens in 6-stable,
as well as 8-stable.

 From what I can see the clone has been freed, but it
remains on the devfs cdevp_list.   Then the next time
devfs_populate_loop() is called, it trips over the bad
entry (cdp->cdp_dirents points to 0xdeadc0dedeadc0de)
See appended kgdb session.

If I trace the code path, it looks like clone_cleanup()
calls destroy_devl().  And destroy_devl() will eventually
call devfs_free() if the si_refcnt is zero.  But I don't
see anything which will get the cdev removed from
the cdevp_list prior to it being freed.

The only code I see which will get the cdev removed from
the cdevp_list() seems to be the "GC any lingering devices"
block in devfs_populate_loop

What am I missing?


Thanks,

Drew


Fatal trap 9: general protection fault while in kernel mode
cpuid = 1; apic id = 01
instruction pointer     = 0x8:0xffffffff803e8780
stack pointer           = 0x10:0xffffffffade623b0
frame pointer           = 0x10:0xffffffffade62400
code segment            = base 0x0, limit 0xfffff, type 0x1b
                         = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 896 (ps)
Dumping 510 MB (2 chunks)
Dumping 510 MB (2 chunks)
Dumping 510 MB (2 chunks)
   chunk 0: 1MB (156 pages) ... ok
   chunk 1: 510MB (130528 pages) 494 478 462 446 430 414 398 382 366 350 
334 318 302 286 270 254 238 222 206 190 174 158 142 126 110 94 78 62 46 
30 14

#0  doadump () at pcpu.h:172
172             __asm __volatile("movq %%gs:0,%0" : "=r" (td));
(kgdb) bt
#0  doadump () at pcpu.h:172
#1  0xffffffff801b8d91 in db_fncall (dummy1=0, dummy2=0, dummy3=0, 
dummy4=0x0) at ../../../ddb/db_command.c:493
#2  0xffffffff801b91e5 in db_command_loop () at 
../../../ddb/db_command.c:408
#3  0xffffffff801bb0ed in db_trap (type=-1377427040, code=0) at 
../../../ddb/db_main.c:222
#4  0xffffffff80468b99 in kdb_trap (type=9, code=0, 
tf=0xffffffffade62300) at ../../../kern/subr_kdb.c:473
#5  0xffffffff806c5d14 in trap_fatal (frame=0xffffffffade62300, 
eva=18446742974557577824)
     at ../../../amd64/amd64/trap.c:660
#6  0xffffffff806c62eb in trap (frame=
       {tf_rdi = -2136471632, tf_rsi = -2136471656, tf_rdx = 
-2401050962867404578, tf_rcx = 1, tf_r8 = -2136471624, tf_r9 = 
-1099151973792, tf_rax = 0, tf_rbx = -1099307447040, tf_rbp = 
-1377426432, tf_r10 = 0, tf_r11 = 4, tf_r12 = 0, tf_r13 = 
-1099086652928, tf_r14 = -1099307447040, tf_r15 = 86032452, tf_trapno = 
9, tf_addr = 0, tf_flags = -2143029088, tf_err = 0, tf_rip = 
-2143385728, tf_cs = 8, tf_rflags = 66071, tf_rsp = -1377426496, tf_ss = 
16}) at ../../../amd64/amd64/trap.c:470
#7  0xffffffff806ad84b in calltrap () at 
../../../amd64/amd64/exception.S:168
#8  0xffffffff803e8780 in devfs_populate_loop (dm=0xffffff000c2b8d00, 
cleanup=0) at ../../../fs/devfs/devfs_devs.c:370
#9  0xffffffff803e8beb in devfs_populate (dm=0xffffff000c2b8d00) at 
../../../fs/devfs/devfs_devs.c:486
#10 0xffffffff803eafab in devfs_lookup (ap=0x0) at 
../../../fs/devfs/devfs_vnops.c:587
#11 0xffffffff80724a2e in VOP_LOOKUP_APV (vop=0xffffffff80948600, 
a=0xffffffffade62630) at vnode_if.c:99
#12 0xffffffff804aadb2 in lookup (ndp=0xffffffffade629c0) at vnode_if.h:56
#13 0xffffffff804abb66 in namei (ndp=0xffffffffade629c0) at 
../../../kern/vfs_lookup.c:216
#14 0xffffffff804c1be2 in vn_open_cred (ndp=0xffffffffade629c0, 
flagp=0xffffffffade6290c, cmode=0,
     cred=0xffffff000009ac00, fdidx=3) at ../../../kern/vfs_vnops.c:183
#15 0xffffffff804b8d64 in kern_open (td=0xffffff00156fe260, 
path=0xffffffff    mode=373490024) at ../../../kern/vfs_syscalls.c:1016
#16 0xffffffff804b9455 in open (td=0xffffffff80a807b0, 
uap=0xffffffffade62bc0) at ../../../kern/vfs_syscalls.c:971
#17 0xffffffff806c6b52 in syscall (frame=
       {tf_rdi = 4218321, tf_rsi = 0, tf_rdx = 0, tf_rcx = 0, tf_r8 = 
140737488348272, tf_r9 = 0, tf_rax = 5, tf_rbx = 5300224, tf_rbp = 
4218321, tf_r10 = 0, tf_r11 = 5300224, tf_r12 = 4218321, tf_r13 = 0, 
tf_r14 = 140737488348272, tf_r15 = 6, tf_trapno = 12, tf_addr = 5300224, 
tf_flags = 0, tf_err = 2, tf_rip = 34369309420, tf_cs = 43, tf_rflags = 
514, tf_rsp = 140737488347528, tf_ss = 35}) at 
../../../amd64/amd64/trap.c:807
#18 0xffffffff806ada48 in Xfast_syscall () at 
../../../amd64/amd64/exception.S:287
#19 0x0000000800920aec in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb) frame 7
#7  0xffffffff806ad84b in calltrap () at 
../../../amd64/amd64/exception.S:168
168             call    trap
Current language:  auto; currently asm
(kgdb) up
#8  0xffffffff803e8780 in devfs_populate_loop (dm=0xffffff000c2b8d00, 
cleanup=0) at ../../../fs/devfs/devfs_devs.c:370
370                     if ((cleanup || !(cdp->cdp_flags & CDP_ACTIVE)) &&
Current language:  auto; currently c
(kgdb) p cdp
$1 = (struct cdev_priv *) 0xffffff0019549a00
(kgdb) p *cdp
$2 = {cdp_c = {si_priv = 0xdeadc0dedeadc0de, si_flags = 3735929054, 
si_atime = {tv_sec = -2401050962867404578,
       tv_nsec = -2401050962867404578}, si_ctime = {tv_sec = 
-2401050962867404578, tv_nsec = -2401050962867404578},
     si_mtime = {tv_sec = -2401050962867404578, tv_nsec = 
-2401050962867404578}, si_uid = 3735929054, si_gid = 3735929054,
     si_mode = 49374, si_cred = 0xdeadc0dedeadc0de, si_drv0 = 
3735929054, si_refcount = -559038242, si_list = {
       le_next = 0xdeadc0dedeadc0de, le_prev = 0xdeadc0dedeadc0de}, 
si_clone = {le_next = 0xdeadc0dedeadc0de,
       le_prev = 0xdeadc0dedeadc0de}, si_children = {lh_first = 
0xdeadc0dedeadc0de}, si_siblings = {
       le_next = 0xdeadc0dedeadc0de, le_prev = 0xdeadc0dedeadc0de}, 
si_parent = 0xdeadc0dedeadc0de,
     si_name = 0xdeadc0dedeadc0de <Address 0xdeadc0dedeadc0de out of 
bounds>, si_drv1 = 0xdeadc0dedeadc0de,
     si_drv2 = 0xdeadc0dedeadc0de, si_devsw = 0xdeadc0dedeadc0de, 
si_iosize_max = -559038242,
ade629c0 "Ñ]@", pathseg=343718984, flags=1,