ZFS freeze/livelock

[Willem Jan Withagen]

On 2010-10-12 17:36, Pawel Jakub Dawidek wrote:
> On Sun, Oct 10, 2010 at 05:34:39PM +0200, Willem Jan Withagen wrote:
>> Hi,
>>
>> Just had my FreeBSD freeze on me with what I would think is sort of an
>> livelock....
>>
>> While I was receiving zfs snapshots on my data pool.
>>
>> Top and systat just kept running,
>> but anything getting near a shell (and perhaps disk-io) ended up in:
>>
>> root at zfs.digiware.nl# gpart create -s gpt da6
>> load: 0.00  cmd: csh 12393 [zfsvfs->z_teardown_inactive_lock] 26.12r
>> 0.00u 0.00s 0% 2480k
>> load: 0.10  cmd: csh 12393 [zfsvfs->z_teardown_inactive_lock] 96.01r
>> 0.00u 0.00s 0% 2480k
>>
>> Trying to execute to execute shutdown -r now had no effect what so ever.
>> Neither did the three-finger salute.
>> (Well at least not in 60 sec I was willing to wait.)
>>
>> Only way out of this situation was hard-reset. And I do have to admit I
>> like ZFS for the speed it recovers after unexpected reboot.
>>
>> To bad there was no alt-ctrl-backspace escape to debugger compiled in.
>> I'll do that with the next kernel, just in case.
>>
>> So the only data point I can give is the ^T output above.
>
> Maybe you still be able to provide backtraces for all processes with
> 'procstat -kk'?
>
> It looks like a deadlock related to 'zfs recv' or maybe unmounting?

System has long rebooted.....

I was no longer ableto start any new programs.
Probably due to pwdbeing on the locked volume.

But I'll this in mind for future time.

And yes I was recv-ing snapshots from my other/working zfs-system.

Having played with that quite some time, I really like that feature.
Although it is not yet robust enough. Perhaps due to the above type problem.

I'm now running it with all debug/witness flags on, but boy does that 
make it slow...

--WjW