kern/73807: panic: mutex nfsd_mtx not owned in NFS server

[Douglas Wells]

>Number:         73807
>Category:       kern
>Synopsis:       panic: mutex nfsd_mtx not owned in NFS server
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Nov 11 10:10:21 GMT 2004
>Closed-Date:
>Last-Modified:
>Originator:     Douglas Wells
>Release:        FreeBSD 5.3
>Organization:
Connection Technologies
>Environment:
System: FreeBSD filbert.contek.com 5.2.1-RELEASE-p11 FreeBSD 5.2.1-RELEASE-p11 #2: Wed Nov 10 16:59:00 EST 2004 root at filbert.contek.com:/var/obj/usr/src.5.2.1/sys/FILBERT i386

	The problem environment is the same as the system above
	except that the OS is FreeBSD 5.3 - cvsup updated (RELENG_5_3)
	November 9 evening. Updated from 5.2.1 via recommended
	buildworld / buildkernel sequence.

>Description:

	The FreeBSD system panics when I attempt to boot a diskless
	Sun 3 that mounts all filesystems from the FreeBSD system.

	The problem began as a NULL pointer dereference when I
	used a custom SMP kernel. The same NULL pointer deference
	occurred when I repeated the problem with a stock GENERIC
	kernel compiled from the same sources. I then ran a variant
	of GENERIC with the following options:
		options	INVARIANT_SUPPORT
		options	INVARIANTS
		options	KDB_TRACE
		options	KDB
		options	DDB

	Repeating the bootload of the Sun 3 led to the following
	panic, which I presume to be due to the same cause as the
	NULL pointer deference: (This information is retyped.)

	    panic: mutex nfsd_mtx not owned at ... nfs_serv.c:4493

	    mtx_assert+5c (c08d97c0, 1, c08160f7, 118d)
	    nfsrv_access+28 (c7217420, 80, c2142180, 0, c1a79190)
	    nfsrv_create+84c (c22142100, c1d62d80, c1a79190, d5469c98, c08d97c0)
	    nfssvc_nfsd+3d5 (c1a79190, c08067a3, 12b, c1c360bc, c1c36000)
	    nfssvc+18c (c1a79190, d5469d14, 2, 0, 296)
	    syscall+213 (2f, 2f, 2f, bf??eec4, 4)

	    thread 100051

	I repeated the experiment several times and generally the
	addresses and thread id vary from run to run. The exceptions
	are the last arg (c08d97c0) to nfsrv_create and the second
	arg (c08067a3) to nfssvc_nfsd.

	Analysis:

	I believe that the problem is due to the system mishandling
	RPC requests for mounts of regular files (mountd(8) -r):
	  - If I remove the -r option to mountd and attempt to
	    boot the Sun 3, the FreeBSD system does not panic (but
	    of course, the Sun 3 does not boot properly).
	  - If I leave the Sun 3 up from having booted from a 5.2.1
	    system, the FreeBSD system panics very soon after it
	    starts up system daemons. (Note that the mountd daemon
	    would not be invoked in this case. The NFS request
	    would use information retained from the previous (5.2.1)
	    boot.)

	Additional notes:
	  - This bootload operationg has previously worked regularly
	    on both 5.1 and 5.2+.
	  - I have modified mountd to support v2 mounts, but I
	    disabled my extensions during at least some of the
	    tests above. These extensions are simple.
	  - Note that the hardware system has 2 AMD Athlon CPUs,
	    but I understand that by not including the SMP kernel
	    option, the OS behaves as though the hardware were
	    uniprocessor.

>How-To-Repeat:

	This happens repeatably whenever I attempt to bootload the
	Sun 3. I presume that sending the NFS server an I/O request
	for any regular file that is being treated as a mount
	point would also trigger this problem.
>Fix:

	None known.

>Release-Note:
>Audit-Trail:
>Unformatted: