sshfs/nfs cause server lockup

Kris Kennaway kris at obsecurity.org
Wed Nov 22 09:06:15 PST 2006


On Wed, Nov 22, 2006 at 05:49:12AM +0000, Chris wrote:
> On a few occasions all different remote servers I have had nfs cause
> servers to stop responding so I stopped  using it all the servers were
> either 6.0 release 6.1 release or 6-stable.
> 
> We recently discovered sshfs which supports cross platform mounting
> server is linux and I mounted on a freebsd 6.1 release using security
> branch up to date.
> 
> it was working fine for around 5 to 6 days with some problems with
> sshfs not updating files that are updated but wasnt compromising the
> stability of the freebsd server I just remounted to keep up to date.
> Then today the linux server had network problems so the sshfs timed
> out and there is 2 dirs I mount, the first mounted fine a bit slow but
> connected but when I ran the command to mount the 2nd dir the server
> stopped responding.
> 
> My 2nd ssh terminal was alive I tried to run top to see if sshfs was
> hanging or something but when I hit enter top didnt run and the 2nd
> terminal was froze, note both terminals didnt timeout and a ircd
> running on the server also did not timeout but the box wasnt listening
> to any new requests, it was responding to pings fine.
> 
> I have a remote reboot facility on the box but no local access and no
> kvm/serial console facility available this is the case for all of my
> servers.  I initially tried a soft reboot which uses ctrl-alt-delete
> but the pings kept replying so I could see the reboot wasn initiated
> indicating some kind of console lockup as well, I then did a hard
> reboot which brought the server back.
> 
> All logs stopped when the first lockup occured so no errors etc.
> recorded bear in mind I have no local access to this machine.  It does
> appear that 6.x has some kind of serious remote mounting bug because I
> never had these nfs problems in freebsd 5.x.
> 
> I would be interested in any thoughts as to what could help me I have
> rebooted the server now with network mpsafe disabled to see if this
> will help it is using a generic kernel with the following changes.

Sounds like your "sshfs" is causing the kernel to deadlock in that
error situation.  You can confirm by enabling DEBUG_LOCKS and
DEBUG_VFS_LOCKS, then breaking to DDB and running 'show lockedvnods'
when the deadlock occurs.

If you're still having problems with NFS on 6.2, we'd much rather you
reported those so that we can investigate and try to fix them.

Kris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20061122/8fa83632/attachment.pgp


More information about the freebsd-stable mailing list