flock incorrectly detects deadlock on 7-stable and current

[Paul Koch]

Hi,

We have been trying to track down a problem with one of our apps which 
does a lot of flock(2) calls.  flock returns errno 11 (Resource 
deadlock avoided) under certain scenarios.  Our app works fine on 
7-Release, but fails on 7-stable and -current.

The problem appears to be when we have at least three processes doing 
flock() on a file, and one is trying to upgrade a shared lock to an 
exclusive lock but fails with a deadlock avoided.

Attached is a simple flock() test program.

a. Process 1 requests and gets a shared lock
b. Process 2 requests and blocks for an exclusive lock
c. Process 3 requests and gets a shared lock
d. Process 3 requests an upgrade to an exclusive lock but fails (errno 
11)

If we change 'd' to
   Process 3 requests unlock, then requests exclusive lock, it works.


The manual page says:

"A shared lock may be upgraded to an exclusive lock, and vice versa, 
simply by specifying the appropriate lock type; this results in the 
previous lock being released and the new lock applied (possibly after 
other processes have gained and released the lock)."

The manual page doesn't mention that flock() can fail with a deadlock.


Our test environment is:
 - 8 core Intel machine running i386 stable
 - 4 core Intel machine running amd64 current (20080508)
 - 4 core Intel machine running amd64 stable  (20080508)
 - 2 core AMD machine running i386 stable (20080418)
 - 2 core AMD machine running i386 stable (20080418)
 - single core (no hyperthreading) i386 stable (20080418)

There appears to have been changes to kern_lockf.c and other stuff 
around the 10th April to do with deadlock detection.  We don't see the 
problem on 6.2-stable, 7-Release, or 7-stable pre ~10th April.

	Paul.