devfs deadlock

[John Baldwin]

On Friday 18 August 2006 15:26, Alex Unleashed wrote:
> Hello,
> 
> Before anything else I'd like to say I'm working on a SoC project for
> Gentoo for which I have to port a sandbox-like application wrapper (in
> userspace) to FreeBSD, which deals with the building and installing of
> software so that nothing gets screwed in the filesystem. It's
> finished, but unfortunately it gets frozen at random points,
> preventing the system from launching new programs or saving to disk,
> which suggested a kernel bug, as was later confirmed looking at the
> waitchannels and debugging. I know there is someone working out issues
> in devfs code in 6.x, so this might also be interesting to him.
> 
> I've been able to reproduce both in 6.1-RELEASE-p3 and 6-STABLE
> (snapshot from August 16th ~01:00 GMT) a deadlock in devfs code which
> leaves the system unable to access the disk. I've come up with some
> interesting debugging info, and it looks to me like there are vnode
> problems while a sx lock is being held.
> 
> My take at it is that the deadlock occurs when a process gets a lock
> on a vnode (tagged "devfs") and another process xlocks an sx lock
> ("devfsmount"). For some reason the one holding the sx lock wants to
> get the lock on the vnode through devfs_allocv(), and the other
> process wants to get the sx lock through devfs_lookup(). From this
> point on, pretty much anything wanting to touch the filesystem waits
> forever on devfs_root() for another vnode flagged as VV_ROOT and
> locked by the process holding the sx lock.
> 
> Patching the devfs code with fixes from -CURRENT didn't work out.

This deadlock is fixed in current as of this:

kib         2006-09-18 13:23:08 UTC

  FreeBSD src repository

  Modified files:
    sys/fs/devfs         devfs.h devfs_devs.c devfs_vfsops.c 
                         devfs_vnops.c 
  Log:
  Resolve the devfs deadlock caused by LOR between devfs_mount->dm_lock and
  vnode lock in devfs_allocv. Do this by temporary dropping dm_lock around
  vnode locking.
  
  For safe operation, add hold counters for both devfs_mount and devfs_dirent,
  and DE_DOOMED flag for devfs_dirent. The facilities allow to continue after
  dropping of the dm_lock, by making sure that referenced memory does not
  disappear.
  
  Reviewed by:    tegge
  Tested by:      kris
  Approved by:    kan (mentor)
  PR:             kern/102335
  
  Revision  Changes    Path
  1.30      +11 -0     src/sys/fs/devfs/devfs.h
  1.47      +12 -1     src/sys/fs/devfs/devfs_devs.c
  1.51      +20 -4     src/sys/fs/devfs/devfs_vfsops.c
  1.134     +70 -11    src/sys/fs/devfs/devfs_vnops.c

and

kib         2006-09-19 14:03:02 UTC

  FreeBSD src repository

  Modified files:
    sys/fs/devfs         devfs_vnops.c 
  Log:
  Fix the bug in rev. 1.134. In devfs_allocv_drop_refs(), when not_found == 2
  and drop_dm_lock is true, no unlocking shall be attempted. The lock is
  already dropped and memory is freed.
  
  Found with:     Coverity Prevent(tm)
  CID:    1536
  Approved by:    pjd (mentor)
  
  Revision  Changes    Path
  1.135     +1 -1      src/sys/fs/devfs/devfs_vnops.c

-- 
John Baldwin