amd64/128101: IO::KQueue with pipes reliably causes kernel panic

Tue Oct 14 20:50:02 UTC 2008

>Number:         128101
>Category:       amd64
>Synopsis:       IO::KQueue with pipes reliably causes kernel panic
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-amd64
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Oct 14 20:50:01 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator:     Matthew Horsfall
>Release:        FreeBSD 6.3-RELEASE-p2
>Organization:
Dynamic Network Services Inc.
>Environment:
FreeBSD hostname 6.3-RELEASE-p2 FreeBSD 6.3-RELEASE-p2 #1: Thu Jun 26 15:21:30 UTC 2008     root at hostname:/usr/obj/usr/src/sys/DELL1950-MPT  amd64

>Description:
With a simple perl script using IO::KQueue, I'm able to (mostly) instantly cause a kernel panic every time.

I've tested this on a Dell 1950 Quad-Core running FreeBSD 6.3-RELEASE-p2, as well as a Dell Optiplex 745 Dual-Core running 7.0-RELEASE-p4 and a Silicon Mechanics Quad-Core Rackform iServ R266 running 6.3-RELEASE-p2.

I was originally having the same problem as http://lists.freebsd.org/pipermail/freebsd-current/2008-April/085017.html and so I began investigating and discovered that I could cause a crash reliably. However, the dump I get with this script is apparently incomplete and different from the other crash:

[GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd".

Unread portion of the kernel message buffer:
12
panic: page fault
cpuid = 2
Uptime: 35m14s
Dumping 4090 MB (3 chunks)
  chunk 0: 1MB (156 pages) ... ok
  chunk 1: 3323MB (850512 pages) 3307 3291 3275 3259 3243 3227 3211 3195 3179 3163 3147 3131 3115 3099 3083 3067 3051 3035 3019 3003 2987 2971 2955 2939 2923 2907 2891 2875 2859 2843 2827 2811 2795 2779 2763 2747 2731 2715 2699 2683 2667 2651 2635 2619 2603 2587 2571 2555 2539 2523 2507 2491 2475 2459 2443 2427 2411 2395 2379 2363 2347 2331 2315 2299 2283 2267 2251 2235 2219 2203 2187 2171 2155 2139 2123 2107 2091 2075 2059 2043 2027 2011 1995 1979 1963 1947 1931 1915 1899 1883 1867 1851 1835 1819 1803 1787 1771 1755 1739 1723 1707 1691 1675 1659 1643 1627 1611 1595 1579 1563 1547 1531 1515 1499 1483 1467 1451 1435 1419 1403 1387 1371 1355 1339 1323 1307 1291 1275 1259 1243 1227 1211 1195 1179 1163 1147 1131 1115 1099 1083 1067 1051 1035 1019 1003 987 971 955 939 923 907 891 875 859 843 827 811 795 779 763 747 731 715 699 683 667 651 635 619 603 587 571 555 539 523 507 491 475 459 443 427 411 395 379 363 347 331 315 299 283 267 251 235 219 203 187 171 155 139 123 107 91 75 59
  43 27 11 ... ok
  chunk 2: 768MB (196608 pages) 753 737 721 705 689 673 657 641 625 609 593 577 561 545 529 513 497 481 465 449 433 417 401 385 369 353 337 321 305 289 273 257 241 225 209 193 177 161 145 129 113 97 81 65 49 33 17 1

#0  doadump () at pcpu.h:172
172             __asm __volatile("movq %%gs:0,%0" : "=r" (td));
(kgdb) backtrace
#0  doadump () at pcpu.h:172
#1  0x0000000000000004 in ?? ()
#2  0xffffffff802486d7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409
#3  0xffffffff80248d71 in panic (fmt=0xffffff001f1a2980 "\b\nÕ\033\001ÿÿÿPç\026+") at /usr/src/sys/kern/kern_shutdown.c:565
#4  0xffffffff803e245f in trap_fatal (frame=0xffffff001f1a2980, eva=18446742978959837704) at /usr/src/sys/amd64/amd64/trap.c:669
#5  0xffffffff803e27dc in trap_pfault (frame=0xffffffffba630840, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:580
#6  0xffffffff803e2a93 in trap (frame=
      {tf_rdi = 0, tf_rsi = 4, tf_rdx = 1, tf_rcx = 1, tf_r8 = 0, tf_r9 = 0, tf_rax = -1098874572592, tf_rbx = 0, tf_rbp = -1098989819520, tf_r10 = -1098874572800, tf_r11 = -1098989819520, tf_r12 = -2141554528, tf_r13 = -1094763207168, tf_r14 = -1094638512976, tf_r15 = -1167914496, tf_trapno = 12, tf_addr = 0, tf_flags = 1108101562372, tf_err = 16, tf_rip = 0, tf_cs = 8, tf_rflags = 66178, tf_rsp = -1167914744, tf_ss = 16}) at /usr/src/sys/amd64/amd64/trap.c:353
#7  0xffffffff803c9b0b in calltrap () at /usr/src/sys/amd64/amd64/exception.S:168
#8  0x0000000000000000 in ?? ()
#9  0xffffffff80224d71 in kqueue_register (kq=0xffffff011b072600, kev=0xffffffffba630a00, td=0xffffff001f1a2980, waitok=1) at /usr/src/sys/kern/kern_event.c:903
#10 0xffffffff80225225 in kern_kevent (td=0xffffff001f1a2980, fd=4, nchanges=1, nevents=0, k_ops=0xffffffffba630b40, timeout=0x0) at /usr/src/sys/kern/kern_event.c:637
#11 0xffffffff80225b60 in kevent (td=0xffffff001f1a2980, uap=0xffffffffba630bc0) at /usr/src/sys/kern/kern_event.c:571
#12 0xffffffff803e3351 in syscall (frame=
      {tf_rdi = 3, tf_rsi = 140737488349776, tf_rdx = 1, tf_rcx = 0, tf_r8 = 0, tf_r9 = 0, tf_rax = 363, tf_rbx = 1, tf_rbp = 7, tf_r10 = 6, tf_r11 = 514, tf_r12 = 6883336, tf_r13 = 0, tf_r14 = 33, tf_r15 = 65535, tf_trapno = 12, tf_addr = 6585448, tf_flags = 0, tf_err = 2, tf_rip = 34372507708, tf_cs = 43, tf_rflags = 582, tf_rsp = 140737488349752, tf_ss = 35}) at /usr/src/sys/amd64/amd64/trap.c:807
#13 0xffffffff803c9d08 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:287
#14 0x0000000800c2d83c in ?? ()
Previous frame inner to this frame (corrupt stack?)

>How-To-Repeat:
Download the scripts at http://hiddenrealms.org/fbsd/ and run ./fbsdkill.pl

Crash is usually instant or it may take a few seconds.
>Fix:

>Release-Note:
>Audit-Trail:
>Unformatted: