misc/147793: cdrom handling, panic, possible race condition in kernel SMP

Fri Jun 11 16:00:08 UTC 2010

>Number:         147793
>Category:       misc
>Synopsis:       cdrom handling, panic, possible race condition in kernel SMP
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Jun 11 16:00:07 UTC 2010
>Closed-Date:
>Last-Modified:
>Originator:     Emanuale Tissani
>Release:        7.3 amd64 SMP
>Organization:
Italcementi
>Environment:
FreeBSD freebsd.acerbis.it 7.3-RELEASE FreeBSD 7.3-RELEASE #0: Sun Mar 21 05:25:24 UTC 2010     root at driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64
>Description:
Environment:
Freebsd 7.3 SMP amd64, 2 processors, running under vmware esx vsphere update 1, 2048 MB ram, one scsi disk, one network adapter, one cdrom, stock distro kernel.

Problem:
When acd0 cdrom device is present in the machine the machine randomly crashes (can run for one day or crash three times in row in six hour span, totally not deterministically, with panic. same behaviour with 7.2 freebsd.

If I remove the cdrom device from the machine, all is rock stable and runs with no problems.
the kern dump seems to support this thesis, pinpointing the crash to some cdrom handling phase:
--------
[root at freebsd /boot/kernel]# kgdb kernel /var/crash/vmcore.4
--------
GNU gdb 6.1.1 [FreeBSD]
..
Unread portion of the kernel message buffer:
acd0: WARNING - READ_TOC tasakcqdu0e:u eW AtRiNmIeNoGu t-  -R EcAoDm_pTlOeCt itnasgk qrueequuee stt idmieroeuctt l-y c
ompleting reque
s
tF adtiarle cttrlayp
 12:
p
aFgaet afla utlrta pw hi1l2e:  ipna gkee rfnaeull tm owdheil
ec piuni dk e=r ne0l;  maopdiec
idc p=u id0 0=
 f1a;u latpi cv iirdt u=a l 0a1d
drfeasusl       t=  v0ixr2t5u8al
afdadurlte scso d=e     0       x=2 5s8u
pefravulti scoord er    e       a=d  dsautpae,r vpiasgoer  nrote apdr edsaetnat,
piangset rnuoctt iporne speoinntt
eri     n=s t0rxu8c:t0ixonf fpfofifnfftfe8r0    5=1 a06x68d:0
xfsftafcfkff fpfo8i0n5t1era     6 6 d
  s t a =c k0 xp1o0i:n0txefrf   f f f f 8 0 0 0 0=5 00bx2100:0
xfrfafmfef fpfo8i0n3tee7r6      7 4 f 0
 f r a=m e0 xp1o0i:n0txefrf     f f f f 0 0  0 1=2 f05x00100:
0xcfofdfef fsefg0m0e0n1t7       6       5=3 ab0a
sec o0dxe0 ,s elgmiemnitt        0      x=f fbfafsfe,  0txy0p,e  l0ixm1ibt
0       x       f       f=f fDfP,L  t0y,p ep r0exs1 b1
,       l       o       n=g  D1P,L  d0e,f 3p2r e0s,  1g,r alno n1g
1p,r odceefs3s2o r0 ,e fglraang s1      =
 pirnotceersrsuoprt  eenflaabglse       d=,  irnetseurmreu,p tI OPeLna b=l e0d,
rceusrurmeen,t  IpOrPoLc e=ss   0
= c1u7r r(esnwti 6p:r otcaessks         q       =u eu1e1)62
 (thraalpd -naudmdboern -       s=t o1r2ag
e)pa
nitcra:p p anguem bfearu        l       t=
1c2pu
id = 0
Uptime: 1h7m14s
Physical memory: 2035 MB
Dumping 1125 MB: 1110 1094 1078 1062 1046 1030 1014 998 982 966 950 934 918 902 886 870 854 838 822 806 790 774 758 742 726 710 694 678 662 646 630 614 598 582 566 550 534 518 502 486 470 454 438 422 406 390 374 358 342 326 310 294 278 262 246 230 214 198 182 166 150 134 118 102 86 70 54 38 22 6
----------------------------------
back
----------------------------------
#0  doadump () at pcpu.h:195
#1  0x0000000000000004 in ?? ()
#2  0xffffffff805285f9 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
#3  0xffffffff80528a02 in panic (fmt=0x104 <Address 0x104 out of bounds>) at /usr/src/sys/kern/kern_shutdown.c:574
#4  0xffffffff807ec7d3 in trap_fatal (frame=0xffffff00012f5000, eva=Variable "eva" is not available.
) at /usr/src/sys/amd64/amd64/trap.c:777
#5  0xffffffff807ecba5 in trap_pfault (frame=0xffffff8000050a70, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:693
#6  0xffffffff807ed4cc in trap (frame=0xffffff8000050a70) at /usr/src/sys/amd64/amd64/trap.c:464
#7  0xffffffff807d610e in calltrap () at /usr/src/sys/amd64/amd64/exception.S:218
#8  0xffffffff8051a66d in _mtx_lock_sleep (m=0xffffff00015a4678, tid=18446742974217801728, opts=Variable "opts" is not available.
) at /usr/src/sys/kern/kern_mutex.c:339
#9  0xffffffff80527b6c in _sema_post (sema=0xffffff00015a4678, file=Variable "file" is not available.
) at /usr/src/sys/kern/kern_sema.c:79
#10 0xffffffff80266264 in ata_completed (context=Variable "context" is not available.
) at /usr/src/sys/dev/ata/ata-queue.c:490
#11 0xffffffff8056232d in taskqueue_run (queue=0xffffff00013ad000) at /usr/src/sys/kern/subr_taskqueue.c:282
#12 0xffffffff80506a92 in ithread_loop (arg=0xffffff0001346ae0) at /usr/src/sys/kern/kern_intr.c:1181
#13 0xffffffff805034e3 in fork_exit (callout=0xffffffff80506930 <ithread_loop>, arg=0xffffff0001346ae0, frame=0xffffff8000050c80) at /usr/src/sys/kern/kern_fork.c:811
#14 0xffffffff807d64ee in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:554
#15 0x0000000000000000 in ?? ()
#16 0x0000000000000000 in ?? ()
#17 0x0000000000000001 in ?? ()
#18 0x0000000000000000 in ?? ()
#19 0x0000000000000000 in ?? ()
#20 0x0000000000000000 in ?? ()
#21 0x0000000000000000 in ?? ()
#22 0x0000000000000000 in ?? ()
#23 0x0000000000000000 in ?? ()
#24 0x0000000000000000 in ?? ()

>How-To-Repeat:
Totally not deterministic.
>Fix:
Quite drastic: remove the cdrom device from the system.

>Release-Note:
>Audit-Trail:
>Unformatted: