amd64/126100: Kernel panic while turning off accounting
Alan Amesbury
amesbury at umn.edu
Wed Jul 30 17:00:08 UTC 2008
>Number: 126100
>Category: amd64
>Synopsis: Kernel panic while turning off accounting
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: freebsd-amd64
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Wed Jul 30 17:00:08 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator: Alan Amesbury
>Release: 7.0-RELEASE-p3 amd64
>Organization:
University of Minnesota
>Environment:
FreeBSD tumbrel.oitsec.umn.edu 7.0-RELEASE-p3 FreeBSD 7.0-RELEASE-p3 #0: Tue Jul 29 17:28:03 CDT 2008 root at tumbrel.oitsec.umn.edu:/usr/obj/usr/src/sys/OITSEC-7-A amd64
>Description:
While shutting off system accounting ('sudo accton'), the system panics. The first crashdump was lost due to insufficient space being available in /var. However, after cleaning up and triggering another panic, I was able to get a good crashdump. kgdb shows:
Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address = 0x10
fault code = supervisor read data, page not present
instruction pointer = 0x8:0xffffffff80554a39
stack pointer = 0x10:0xffffffffb48078d0
frame pointer = 0x10:0xffffff0091012700
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 35491 (accton)
trap number = 12
panic: page fault
cpuid = 1
Uptime: 17h24m2s
Physical memory: 16369 MB
Dumping 918 MB: 903 887 871 855 839 823 807 791 775 759 743 727 711 695 679 663 647 631 615 599 583 567 551 535 519 503 487 471 455 439 423 407 391 375 359 343 327 311 295 279 263 247 231 215 199 183 167 151 135 119 103 87 71 55 39 23 7
#0 doadump () at pcpu.h:194
194 __asm __volatile("movq %%gs:0,%0" : "=r" (td));
(kgdb) list
189 static __inline struct thread *
190 __curthread(void)
191 {
192 struct thread *td;
193
194 __asm __volatile("movq %%gs:0,%0" : "=r" (td));
195 return (td);
196 }
197 #define curthread (__curthread())
198
(kgdb) backtrace *0xffffffff80554a39
#0 doadump () at pcpu.h:194
#1 0x0000000000000000 in ?? ()
#2 0xffffffff803bd1b6 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409
#3 0xffffffff803bd471 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:563
#4 0xffffffff805ce784 in trap_fatal (frame=0xffffffffb4807820, eva=16) at /usr/src/sys/amd64/amd64/trap.c:724
#5 0xffffffff805ce9df in trap_pfault (frame=0xffffffffb4807820, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:641
#6 0xffffffff805cf31b in trap (frame=0xffffffffb4807820) at /usr/src/sys/amd64/amd64/trap.c:410
#7 0xffffffff805b60ee in calltrap () at /usr/src/sys/amd64/amd64/exception.S:169
#8 0xffffffff80554a39 in mac_bsdextended_check_vp (cred=0xffffff0091012700, vp=0x0, acc_mode=128) at vnode_if.h:285
#9 0xffffffff80555173 in mac_bsdextended_check_system_acct (cred=Variable "cred" is not available.
) at /usr/src/sys/security/mac_bsdextended/mac_bsdextended.c:447
#10 0xffffffff8055051f in mac_check_system_acct (cred=0xffffff0091012700, vp=0x0) at /usr/src/sys/security/mac/mac_system.c:136
#11 0xffffffff80389c65 in acct (td=0xffffff00231656a0, uap=0xffffffffb4807be0) at /usr/src/sys/kern/kern_acct.c:238
#12 0xffffffff805ced56 in syscall (frame=0xffffffffb4807c70) at /usr/src/sys/amd64/amd64/trap.c:852
#13 0xffffffff805b62fb in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:290
#14 0x00000003ef5e03ac in ?? ()
Previous frame inner to this frame (corrupt stack?)
We use a custom kernel config (below). We do *NOT* have any filesystems mounted with ACL or MAC support (i.e., the "-a" and "-l" options in 'tunefs') on the system that panics. I've other systems which *do* make use of those features, though, and we try to keep kernels identical across platforms.
Here's the kernel config:
# $Id: OITSEC-7-A,v 1.2 2008/01/04 22:01:46 amesbury Exp $
machine amd64
cpu HAMMER
ident OITSEC-7-A
# To statically compile in device wiring instead of /boot/device.hints
#hints "GENERIC.hints" # Default places to look for devices.
makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols
options SCHED_ULE # ULE scheduler
options PREEMPTION # Enable kernel thread preemption
options INET # InterNETworking
options INET6 # IPv6 communications protocols
options SCTP # Stream Control Transmission Protocol
options FFS # Berkeley Fast Filesystem
options SOFTUPDATES # Enable FFS soft updates support
options UFS_ACL # Support for access control lists
options UFS_DIRHASH # Improve performance on big directories
options UFS_GJOURNAL # Enable gjournal-based UFS journaling
options MD_ROOT # MD is a potential root device
options NFSCLIENT # Network Filesystem Client
options NFSSERVER # Network Filesystem Server
options NFS_ROOT # NFS usable as /, requires NFSCLIENT
options MSDOSFS # MSDOS Filesystem
options CD9660 # ISO 9660 Filesystem
options PROCFS # Process filesystem (requires PSEUDOFS)
options PSEUDOFS # Pseudo-filesystem framework
options GEOM_PART_GPT # GUID Partition Tables.
options GEOM_LABEL # Provides labelization
options COMPAT_43TTY # BSD 4.3 TTY compat [KEEP THIS!]
options COMPAT_FREEBSD6 # Compatible with FreeBSD6
options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI
options KTRACE # ktrace(1) support
options SYSVSHM # SYSV-style shared memory
options SYSVMSG # SYSV-style message queues
options SYSVSEM # SYSV-style semaphores
options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions
options KBD_INSTALL_CDEV # install a CDEV entry in /dev
options ADAPTIVE_GIANT # Giant mutex is adaptive.
options STOP_NMI # Stop CPUS using NMI instead of IPI
options AUDIT # Security event auditing
# To make an SMP kernel, the next two lines are needed
options SMP # Symmetric MultiProcessor Kernel
# CPU frequency control
device cpufreq
# Bus support.
device acpi
device pci
# Floppy drives
device fdc
# ATA and ATAPI devices
device ata
device atadisk # ATA disk drives
device ataraid # ATA RAID drives
device atapicd # ATAPI CDROM drives
device atapifd # ATAPI floppy drives
device atapist # ATAPI tape drives
options ATA_STATIC_ID # Static device numbering
# SCSI Controllers
device ahc # AHA2940 and onboard AIC7xxx devices
options AHC_REG_PRETTY_PRINT # Print register bitfields in debug
# output. Adds ~128k to driver.
device ahd # AHA39320/29320 and onboard AIC79xx devices
options AHD_REG_PRETTY_PRINT # Print register bitfields in debug
# output. Adds ~215k to driver.
device amd # AMD 53C974 (Tekram DC-390(T))
device mpt # LSI-Logic MPT-Fusion
# SCSI peripherals
device scbus # SCSI bus (required for SCSI)
device ch # SCSI media changers
device da # Direct Access (disks)
device sa # Sequential Access (tape etc)
device cd # CD
device pass # Passthrough device (direct SCSI access)
device ses # SCSI Environmental Services (and SAF-TE)
# RAID controllers interfaced to the SCSI subsystem
device amr # AMI MegaRAID
device asr # DPT SmartRAID V, VI and Adaptec SCSI RAID
device ciss # Compaq Smart RAID 5*
device dpt # DPT Smartcache III, IV - See NOTES for options
device hptmv # Highpoint RocketRAID 182x
device iir # Intel Integrated RAID
device ips # IBM (Adaptec) ServeRAID
device mly # Mylex AcceleRAID/eXtremeRAID
device twa # 3ware 9000 series PATA/SATA RAID
# RAID controllers
device aac # Adaptec FSA RAID
device aacp # SCSI passthrough for aac (requires CAM)
device ida # Compaq Smart RAID
device mfi # LSI MegaRAID SAS
device mfip # LSI MegaRAID pass-through interface (requires CAM)
device twe # 3ware ATA RAID
# atkbdc0 controls both the keyboard and the PS/2 mouse
device atkbdc # AT keyboard controller
device atkbd # AT keyboard
device psm # PS/2 mouse
device kbdmux # keyboard multiplexer
device vga # VGA video card driver
device splash # Splash screen and screen saver support
# syscons is the default console driver, resembling an SCO console
device sc
device agp # support several AGP chipsets
# Power management support (see NOTES for more options)
#device apm
# Serial (COM) ports
device sio # 8250, 16[45]50 based serial ports
device uart # Generic UART driver
# If you've got a "dumb" serial or parallel PCI card that is
# supported by the puc(4) glue driver, uncomment the following
# line to enable it (connects to the sio and/or ppc drivers):
#device puc
# PCI Ethernet NICs.
device em # Intel PRO/1000 adapter Gigabit Ethernet Card
device ixgb # Intel PRO/10GbE Ethernet Card
device txp # 3Com 3cR990 (``Typhoon'')
device vx # 3Com 3c590, 3c595 (``Vortex'')
# PCI Ethernet NICs that use the common MII bus controller code.
# NOTE: Be sure to keep the 'device miibus' line in order to use these NICs!
device miibus # MII bus support
device bce # Broadcom BCM5706/BCM5708 Gigabit Ethernet
device bfe # Broadcom BCM440x 10/100 Ethernet
device bge # Broadcom BCM570xx Gigabit Ethernet
device dc # DEC/Intel 21143 and various workalikes
device fxp # Intel EtherExpress PRO/100B (82557, 82558)
device lge # Level 1 LXT1001 gigabit Ethernet
device msk # Marvell/SysKonnect Yukon II Gigabit Ethernet
device nfe # nVidia nForce MCP on-board Ethernet
device nge # NatSemi DP83820 gigabit Ethernet
device re # RealTek 8139C+/8169/8169S/8110S
device rl # RealTek 8129/8139
device sis # Silicon Integrated Systems SiS 900/SiS 7016
device sk # SysKonnect SK-984x & SK-982x gigabit Ethernet
device tx # SMC EtherPower II (83c170 ``EPIC'')
device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'')
# Pseudo devices.
device loop # Network loopback
device random # Entropy device
device ether # Ethernet support
device tun # Packet tunnel.
device pty # Pseudo-ttys (telnet etc)
device md # Memory "disks"
device gif # IPv6 and IPv4 tunneling
device faith # IPv6-to-IPv4 relaying (translation)
# The `bpf' device enables the Berkeley Packet Filter.
# Be aware of the administrative consequences of enabling this!
# Note that 'bpf' is required for DHCP.
device bpf # Berkeley packet filter
# USB support
device uhci # UHCI PCI->USB interface
device ohci # OHCI PCI->USB interface
device ehci # EHCI PCI->USB interface (USB 2.0)
device usb # USB Bus (required)
#device udbp # USB Double Bulk Pipe devices
device ugen # Generic
device uhid # "Human Interface Devices"
device ukbd # Keyboard
device ulpt # Printer
device umass # Disks/Mass storage - Requires scbus and da
device ums # Mouse
# FireWire support
device firewire # FireWire bus code
device sbp # SCSI over FireWire (Requires scbus and da)
device fwe # Ethernet over FireWire (non-standard!)
options ALTQ
options ALTQ_CBQ
options ALTQ_HFSC
options ALTQ_PRIQ
device pf
device pflog
options ZERO_COPY_SOCKETS
options MAC
options MAC_BSDEXTENDED
options MAC_PARTITION
options HZ=1000
options SC_HISTORY_SIZE=1000
options SC_KERNEL_CONS_ATTR=(FG_YELLOW|BG_BLACK)
options SC_KERNEL_CONS_REV_ATTR=(FG_BLACK|BG_RED)
options DEVICE_POLLING
options AUTO_EOI_1
options INCLUDE_CONFIG_FILE
device carp
>How-To-Repeat:
Turn off accounting on an FreeBSD 7.0-RELEASE-p3/amd64 host that has mac_bsdextended loaded.
>Fix:
I'm unsure of the fix, but the problem appears to be related to mac_bsdextended. Our kernel has it compiled in. On a similar system (somewhat different kernel, but one lacking mac_bsdextended), 'sudo accton' doesn't trigger a panic. However, once you load the mac_bsdextended and try the command again, that system panics.
>Release-Note:
>Audit-Trail:
>Unformatted:
More information about the freebsd-amd64
mailing list