ZFS deadlock

[LI Xin]

Johan Ström wrote:
> Hello
> 
> A box of mine running RELENG_7_0 and ZFS over a couple of disks (6 
> disks, 3 mirrors) seems to have gotten stuck. From Ctrl-T:
> 
> load: 0.50  cmd: zsh 40188 [zfs:&buf_hash_table.ht_locks[i].ht_lock] 
> 0.02u 0.04s 0% 3404k
> load: 0.43  cmd: zsh 40188 [zfs:&buf_hash_table.ht_locks[i].ht_lock] 
> 0.02u 0.04s 0% 3404k
> load: 0.10  cmd: zsh 40188 [zfs:&buf_hash_table.ht_locks[i].ht_lock] 
> 0.02u 0.04s 0% 3404k
> load: 0.10  cmd: zsh 40188 [zfs:&buf_hash_table.ht_locks[i].ht_lock] 
> 0.02u 0.04s 0% 3404k
> load: 0.11  cmd: zsh 40188 [zfs:&buf_hash_table.ht_locks[i].ht_lock] 
> 0.02u 0.04s 0% 3404k
> 
> Worked for a while then that stopped working too (was over ssh). When 
> trying a local login i only got
> 
> load: 0.09  cmd: login 1611 [zfs] 0.00u 0.00s 0% 208k
> 
> I found one post like this earlier (by Xin LI), but nobody seemed to 
> have replied...
> in my current conf, I think my kmem/kmem_max is at 512Mb (not sure 
> though, since I've edited my file yesterday for next reboot), with 2G of 
> system RAM.. Normally I'd run kmem(max) 1G (with arcsize of 512M. 
> currently it is at default), but since I just got back to 2G total mem 
> after some hardware problems I've been runnig at those lows (1G total is 
> kindof tight with zfs..)
> 
> Well, just wanted to report... The box is not totally dead yet, ie I can 
> still do Ctrl-T on console, but thats it.. I don't really know what more 
> I can do so.. I don't have KDB/DDB.
> I'll wait another hour or so before I hard reboot it, unless it 
> "unlocks" or if anyone have any suggestions.

The key is to increase your kmem and prevent it from being exhausted.  I 
think more recent OpenSolaris's ZFS code has some improvements but I do 
not have spare devices at hand to test and debug :(

Maybe pjd@ would get a new import at some point?  I have cc'ed him.

Cheers,
-- 
Xin LI <delphij at delphij.net>	http://www.delphij.net/
FreeBSD - The Power to Serve!

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 249 bytes
Desc: OpenPGP digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20080408/d9a58dae/signature.pgp