kern/119753: tmpfs causes a panic at shutdown

Thu Jan 17 19:00:11 UTC 2008

>Number:         119753
>Category:       kern
>Synopsis:       tmpfs causes a panic at shutdown
>Confidential:   no
>Severity:       serious
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Jan 17 19:00:01 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator:     Yoshihiro Ota
>Release:        FreeBSD 7-BETA4/RC1
>Organization:
>Environment:
FreeBSD 7.0-RC1 ... GENERIC ... i386
>Description:
This started happening around 12/15/2007.

The cause of panic is force unmount of tmpfs at shutdown/reboot.  I am not sure if this happens at only shutdown/reboot time or anytime with umount -f.

I do not yet have a concrete way to reproduced this problem, but:
1. Tmpfs needs to be mounted at shutdown,
2. No read/write is required on tmpfs to cause this panic.  I.e., I mounted on /mnt/tmpfs and never used it, but it caused a panic.
3. Some activities on the system is required.  If I shutdown immidately after system comes up, it never caused a panic.  It frequently crashes after rsyncing lots of  files.

One this I am sure is that I never seen the panic once I started doing "umount -t tmpfs" before shutdown.

I narrowed down to tmpfs_root.  It never retuend once called from dounmount VFS_ROOT.  The below is where tmpfs_root is called.

int
dounmount(mp, flags, td)
	struct mount *mp;
	int flags;
	struct thread *td;
{
..
	/*
	 * For forced unmounts, move process cdir/rdir refs on the fs root
	 * vnode to the covered vnode.  For non-forced unmounts we want
	 * such references to cause an EBUSY error.
	 */
	if ((flags & MNT_FORCE) &&
--->	    VFS_ROOT(mp, LK_EXCLUSIVE, &fsrootvp, td) == 0) { <---
		if (mp->mnt_vnodecovered != NULL)
			mountcheckdirs(fsrootvp, mp->mnt_vnodecovered);
		if (fsrootvp == rootvnode) {
			vrele(rootvnode);
			rootvnode = NULL;
		}
		vput(fsrootvp);
	}

# kgdb /usr/obj/usr/src/sys/GENERIC/kernel.debug /var/crash/vmcore.3 
[GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so:
Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd".

Unread portion of the kernel message buffer:
/etc/rc.shutdown: WARNING: $samba_enable is not set properly - see rc.conf(5).
Writing entropy file:..
syslod: exiting on signal 15
Waiting (max 60 seconds) for system process `vnlru' to stop...done
Waiting (max 60 seconds) for system process `bufdaemon' to stop...done
Wait
iSnygn c(imnagx  dis60k s,s evcnoonddess)  rfeomra isynsitneg.m. .p3r
ocess `syncer' to stop...3 3 1 1 1 1 0 0 0 0 done
All buffers synced.
GEOM_ELI: Detached da0.eli on last close.
acpi_ec0: warning: EC done before starting event wait
GEOM_ELI: Detached ad4s2f.eli on last close.

Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x188
fault code              = supervisor read, page not present
instruction pointer     = 0x20:0xc07479d4
stack pointer           = 0x28:0xe3c71b1c
frame pointer           = 0x28:0xe3c71b34
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 1 (init)
trap number             = 12
panic: page fault
cpuid = 0
Uptime: 1h24m11s
Physical memory: 2025 MB
Dumping 155 MB: 140 124 108 92 76 60 44 28 12

#0  doadump () at pcpu.h:195
195     pcpu.h: No such file or directory.
        in pcpu.h
(kgdb) where             
#0  doadump () at pcpu.h:195
#1  0xc0753e87 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409
#2  0xc0754149 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:563
#3  0xc0a689ec in trap_fatal (frame=0xe3c71adc, eva=392)
    at /usr/src/sys/i386/i386/trap.c:899
#4  0xc0a68c70 in trap_pfault (frame=0xe3c71adc, usermode=0, eva=392)
    at /usr/src/sys/i386/i386/trap.c:812
#5  0xc0a6961c in trap (frame=0xe3c71adc) at /usr/src/sys/i386/i386/trap.c:490
#6  0xc0a4f5ab in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#7  0xc07479d4 in _mtx_lock_sleep (m=0xc8abfa18, tid=3306133088, opts=0, 
    file=0x0, line=0) at /usr/src/sys/kern/kern_mutex.c:335
#8  0xc07cc94f in vgone (vp=0xc8abf990) at /usr/src/sys/kern/vfs_subr.c:2471
#9  0xc5742a79 in ?? ()
#10 0xc8abf990 in ?? ()
#11 0xc55a9a70 in ?? ()
#12 0x00000000 in ?? ()
#13 0xc5743c20 in ?? ()
#14 0x0000016c in ?? ()
#15 0xc5746074 in ?? ()
#16 0x00003002 in ?? ()
#17 0xc8abf990 in ?? ()
#18 0xe3c71bf0 in ?? ()
#19 0x00000004 in ?? ()
---Type  to continue, or q  to quit---
#20 0xc50f9660 in ?? ()
#21 0xe3c71ba8 in ?? ()
#22 0xc57420b4 in ?? ()
#23 0xc55a9a70 in ?? ()
#24 0xc5746000 in ?? ()
#25 0x00001002 in ?? ()
#26 0xe3c71bf0 in ?? ()
#27 0xc50f9660 in ?? ()
#28 0xc569d550 in ?? ()
#29 0xe3c71c00 in ?? ()
#30 0xc07c6ac1 in dounmount (mp=0xc55a9a70, flags=-982228992, td=0x10)
    at /usr/src/sys/kern/vfs_mount.c:1270
Previous frame inner to this frame (corrupt stack?)
(kgdb)
>How-To-Repeat:
# echo 'tmpfs /mnt/tmp tmpfs rw 0 0' >> /etc/fstab
# mkdir /mnt/tmp
# mount /mnt/tmp

# some activities...

# shutdown -p now

>Fix:

>Release-Note:
>Audit-Trail:
>Unformatted: