sshfs/nfs cause server lockup

Chris chrcoluk at gmail.com
Wed Nov 22 21:25:55 PST 2006


On 22/11/06, Kris Kennaway <kris at obsecurity.org> wrote:
> On Wed, Nov 22, 2006 at 05:49:12AM +0000, Chris wrote:
> > On a few occasions all different remote servers I have had nfs cause
> > servers to stop responding so I stopped  using it all the servers were
> > either 6.0 release 6.1 release or 6-stable.
> >
> > We recently discovered sshfs which supports cross platform mounting
> > server is linux and I mounted on a freebsd 6.1 release using security
> > branch up to date.
> >
> > it was working fine for around 5 to 6 days with some problems with
> > sshfs not updating files that are updated but wasnt compromising the
> > stability of the freebsd server I just remounted to keep up to date.
> > Then today the linux server had network problems so the sshfs timed
> > out and there is 2 dirs I mount, the first mounted fine a bit slow but
> > connected but when I ran the command to mount the 2nd dir the server
> > stopped responding.
> >
> > My 2nd ssh terminal was alive I tried to run top to see if sshfs was
> > hanging or something but when I hit enter top didnt run and the 2nd
> > terminal was froze, note both terminals didnt timeout and a ircd
> > running on the server also did not timeout but the box wasnt listening
> > to any new requests, it was responding to pings fine.
> >
> > I have a remote reboot facility on the box but no local access and no
> > kvm/serial console facility available this is the case for all of my
> > servers.  I initially tried a soft reboot which uses ctrl-alt-delete
> > but the pings kept replying so I could see the reboot wasn initiated
> > indicating some kind of console lockup as well, I then did a hard
> > reboot which brought the server back.
> >
> > All logs stopped when the first lockup occured so no errors etc.
> > recorded bear in mind I have no local access to this machine.  It does
> > appear that 6.x has some kind of serious remote mounting bug because I
> > never had these nfs problems in freebsd 5.x.
> >
> > I would be interested in any thoughts as to what could help me I have
> > rebooted the server now with network mpsafe disabled to see if this
> > will help it is using a generic kernel with the following changes.
>
> Sounds like your "sshfs" is causing the kernel to deadlock in that
> error situation.  You can confirm by enabling DEBUG_LOCKS and
> DEBUG_VFS_LOCKS, then breaking to DDB and running 'show lockedvnods'
> when the deadlock occurs.
>
> If you're still having problems with NFS on 6.2, we'd much rather you
> reported those so that we can investigate and try to fix them.
>
> Kris
>
>
>

Ok thanks, I will make sure this box is updated to 6.2 when it hits
release, if I enable the options in the kernel I will need local
access to use ddb?

Chris


More information about the freebsd-stable mailing list