[PATCH] L2ARC deadlock

[Pawel Jakub Dawidek]

On Sun, Aug 07, 2011 at 01:52:45AM -0700, Xin LI wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> Hi,
> 
> I have observed this on a test system which have both L2ARC and ZIL
> device installed, and several vdev groups.  This can be triggered by
> removing the l2arc and zil device (detaching l2arc would fail), blocking
> at 'spa_namespace_lock'
> 
> After reboot, the pool can not be imported.  'zpool import' shows the
> zpool command stuck with 'spa_namespace_lock'.
> 
> I have a theory about the issue but the patch was not tested, maybe
> someone who is more familiar with the code can shed me some light?

Yes, your theory is correct, but the problem is wider (there are other
cases it might deadlock as we discussed with Justin (CCed)).

The reason for it is that we have two GEOM classes in ZFS: ZVOL and
VDEV_GEOM. ZVOL calls from GEOM into ZFS and VDEV_GEOM calls from ZFS
into GEOM. This creates lock order reversals. While working on ZFSv28 I
cleaned up locking, so that locks are always acquired in the following
order:

	zfsdev_state_lock
	g_topology_lock
	spa_namespace_lock

Unfortunately I missed spa_config lock, which is special and not handled
by WITNESS and the problem you are seeing is indeed related to that lock
(reversal between spa_namespace_lock and spa_config lock).

I just tried to simplify the locking by eliminating the
zfsdev_state_lock altogether and replacing it with the
spa_namespace_lock.

Xin, could you try the following patch:

	http://people.freebsd.org/~pjd/patches/zfsdev_state_lock.patch

Justin, what do you think?

-- 
Pawel Jakub Dawidek                       http://www.wheelsystems.com
FreeBSD committer                         http://www.FreeBSD.org
Am I Evil? Yes, I Am!                     http://yomoli.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/zfs-devel/attachments/20110808/604942c8/attachment.pgp