8.1R possible zfs snapshot livelock?

[Charles Sprickman]

On Tue, 17 May 2011, Andriy Gapon wrote:

> on 17/05/2011 15:23 Jeremy Chadwick said the following:
>> So for Charles' situation, the next time it happens what would be useful
>> for him to provide?  The best I could come up with was to induce doadump
>> then reboot to get the system up/working again, and then use kgdb
>> after-the-fact.
>
> This is one of the best things to do, if possible.
> In this case all the potentially useful info would be preserved.

Will do.  Just have to verify I understand the "break to debugger" stuff 
with serial consoles and ensure it's not easy to accidentally trigger 
(vauge memories of Sun boxes doing funny things if someone messes with the 
console).

We're also going to start in on upgrading to 8.2 and see what that brings. 
Still sounds like a good idea to be able to force a dump if things lock up 
like this again, regardless of what we're running.

> Less drastic approach to hanged I/O debugging is to find out where
> processes/threads are actually stuck.  E.g. using procstat -kk.

Odd you say that because we've got an old 32-bit 8.1 box that's running 
spamassassin and some devel stuff that looks like it's getting a little 
wedged:

   PID USERNAME  THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
     6 root        4  -8    -     0K    36K tx->tx 126.0H 76.37% zfskern

And I'm not sure "procstat" is meant for this, but the output is 
interesting:

[root at h22 /home/spork]# procstat -k 6
   PID    TID COMM             TDNAME           KSTACK
     6 100053 zfskern          arc_reclaim_thre mi_switch sleepq_switch 
sleepq_timedwait _cv_timedwait arc_reclaim_thread fork_exit 
fork_trampoline
     6 100054 zfskern          l2arc_feed_threa mi_switch sleepq_switch 
sleepq_timedwait _cv_timedwait l2arc_feed_thread fork_exit fork_trampoline
     6 100093 zfskern          txg_thread_enter mi_switch sleepq_switch 
sleepq_wait _cv_wait txg_thread_wait txg_quiesce_thread fork_exit 
fork_trampoline
     6 100094 zfskern          txg_thread_enter mi_switch sleepq_switch 
sleepq_timedwait _cv_timedwait txg_thread_wait txg_sync_thread fork_exit 
fork_trampoline

Makes me curious about the patch mm@ has for 8.2 here:

http://blog.vx.sk/archives/24-Backported-patches-for-FreeBSD-82-RELEASE.html 
(item "c" in the list)

Anyhow, thank you *all* for an interesting discussion.  We do want to 
forge ahead with using snapshots since it's a really nice luxury, 
especially on boxes with lots of jails.  Makes it very easy to roll things 
back without having to go to the tapes.

Thanks,

Charles

> -- 
> Andriy Gapon
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
>