Recurring problem: processes block accessing UFS file system
Don Lewis
truckman at FreeBSD.org
Thu Jan 12 15:31:45 PST 2006
On 12 Jan, Don Lewis wrote:
> On 11 Jan, Denis Shaposhnikov wrote:
>> Hi!
>>
>>>>>>> "Don" == Don Lewis <truckman at FreeBSD.org> writes:
>>
>> Don> Are you using any unusual file systems, such as nullfs or
>> Don> unionfs?
>>
>> >> Yes, I'm use a lots of nullfs. This is a host system for about 20
>> >> jails with nullfs mounted ro system:
>>
>> Don> That would be my guess as to the cause of the problem. Hopefully
>> Don> DEBUG_VFS_LOCKS will help pinpoint the bug.
>>
>> I've got the problem again. Now I have debug kernel and crash
>> dump. That is an output from the kdb. Do you need any additional
>> information which I can get from the crash dump?
> Process 33016 is executing rmdir(). While doing the lookup, it is
> holding a lock on vnode 0xc6c07bf4 and attempting to lock vnode
> c6bed3fc. Vnode 0xc6c07bf4 should be the parent directory of c6bed3fc.
>
> Process 546 is executing open(). While doing the lookup, it is holding
> a lock on vnode 0xc6bed3fc while attempting to lock vnode c6c07bf4.
> Vnode 0xc6bed3fc should be the parent directory of c6c07bf4, but this is
> inconsistent with the previous paragraph.
>
> This situation should not be possible. Using kgdb on your saved crash
> dump, print "fmode" and "*ndp" in the vn_open_cred() stack frame of
> process 546, and "*nd" in the kern_rmdir() stack frame of process 33016.
> The path names being looked up may be helpful.
>
> Are there any symbolic links in the path names? If so, what are the
> link contents?
>
> Are either of these processes jailed? If so, same or different jails?
>
> What are inodes 2072767 and 2072795 on ad4s1g?
>
> Are you using snapshots?
I just thought of another possible cause for this problem. Is is
possible that you have any hard links to directories in the file system
on ad4s1g? That could put a loop in the directory tree and mess up the
normal parent-child relationship that we rely on to avoid deadlocks.
More information about the freebsd-stable
mailing list