NFS processes locking up!!

[Mark Morley]

Hi all,

We have an NFS server (amd64) running FreeBSD 6.1-STABLE.  It serves a dozen
or so clients which are a mix of FreeBSD 4.11 and 6.1-STABLE.  All NFS traffic
is on a dedicated gigabit switched network.

Periodically we have a problem where it will stop serving up files.  Running 'ps'
on the server shows a number of processes stuck in the 'D' state -- "a process in
disk (or other short term, uninter-ruptible) wait".

Usually this includes all the nfsd processes as well as any others that are trying
to access the same disk drive.  Any commands issued like 'du', 'sync', etc. go into
the same state and never exit.  It is impossible to kill any of these processes.

We can pretty much force this to happen by running a large 'find' or something
similar on the exported file system, although it will happen itself eventually
without any such commands being run.

Our only option (as far as we can tell) is to reboot the server, which results in
a very long fsck period (it's over a terrabyte of disk space).

This doesn't seem to be a hardware issue.  This is a brand new server in all respects
(all new hardware, new RAID) and we saw the exact same issue on the machine that it
replaced (which was running 4.11 on i386).

Any thoughts on this?  Any more info I should provide?

Mark
--
Mark Morley
Owner / Administrator
Islandnet.com