FreeBSD-9.1: machine reboots during snapshot creation, LORs found

[Andre Albsmeier]

Each day at 5:15 we are generating snapshots on various machines.
This used to work perfectly under 7-STABLE for years but since
we started to use 9.1-STABLE the machine reboots in about 10%
of all cases.

After rebooting we find a new snapshot file which is a bit
smaller than the good ones and with different permissions
It does not succeed a fsck. In this example it is the one
whose name is beginning with s3:

-r--r-----   1 root  operator  snapshot 72802894528 29 May 05:15 s2-2013.05.28-03.15.04
-r--------   1 root  operator  snapshot 72802893824 29 May 05:15 s3-2013.05.29-03.15.03
-r--r-----   1 root  operator  snapshot 72802894528 28 May 14:22 s4-2013.05.23-06.38.44
-r--r-----   1 root  operator  snapshot 72802894528 28 May 14:22 s5-2013.05.24-03.15.03
-r--r-----   1 root  operator  snapshot 72802894528 28 May 14:22 s6-2013.05.25-03.15.03

After enabling DIAGNOSTIC, WITNESS and INVARIANTS in the kernel
I see the following LORs (mksnap_ffs starts exactly at 5:15):

May 29 05:15:00 <kern.crit> palveli kernel: lock order reversal:
May 29 05:15:00 <kern.crit> palveli kernel: 1st 0xc2371da8 ufs (ufs) @ /src/src-9/sys/kern/vfs_mount.c:1240
May 29 05:15:00 <kern.crit> palveli kernel: 2nd 0xc2371ec4 devfs (devfs) @ /src/src-9/sys/ufs/ffs/ffs_vfsops.c:1414
May 29 05:15:04 <kern.crit> palveli kernel: lock order reversal:
May 29 05:15:04 <kern.crit> palveli kernel: 1st 0xc228471c snaplk (snaplk) @ /src/src-9/sys/ufs/ufs/ufs_vnops.c:976
May 29 05:15:04 <kern.crit> palveli kernel: 2nd 0xc22f25e4 ufs (ufs) @ /src/src-9/sys/ufs/ffs/ffs_snapshot.c:1626

Unfortunatley no corefiles are being generated ;-(.

I have checked and even rebuilt the (UFS1) fs in question
from scratch. I have also seen this happen on an UFS2 on
another machine and on a third one when running "dump -L"
on a root fs.

Any hints of how to proceed?

	-Andre