[Bug 229958] ctld / zfs deadlock with 'zfs rename ...'

bugzilla-noreply at freebsd.org bugzilla-noreply at freebsd.org
Wed Dec 12 18:21:57 UTC 2018


https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229958

--- Comment #8 from Dave Baukus <daveb at spectralogic.com> ---
(In reply to Andriy Gapon from comment #7)
I have several deadlocks that all involve zvol and the spa_namespace_lock; the
following does not involve a rename:

spa_namespace_lock lock holder blocked on the dp->dp_config_rwlock
------------------------------------------------------------------
Thread 1745 (Thread 100163):
(struct thread *)0xfffff80003cdc580, tid 100163
   geom :: (struct proc *)0xfffff80003d6ba50, pid 13

STACK:
#0  sched_switch (td=0xfffff80003cdc580, newtd=0xfffff80003acf580,
flags=<optimized out>) at sys/kern/sched_ule.c:2112
#1  0xffffffff80513d6f in mi_switch (flags=260, newtd=0x0) at
sys/kern/kern_synch.c:439
#2  0xffffffff8055e04d in sleepq_switch (wchan=0xfffff80e53a8ecc8, pri=0) at
sys/kern/subr_sleepqueue.c:613
#3  0xffffffff8055def3 in sleepq_wait (wchan=0xfffff80e53a8ecc8, pri=0) at
sys/kern/subr_sleepqueue.c:692
#4  0xffffffff804a4b73 in _cv_wait (cvp=0xfffff80e53a8ecc8, lock=<optimized
out>) at sys/kern/kern_condvar.c:146
#5  0xffffffff8183a54b in rrw_enter_read_impl (rrl=0xfffff80e53a8eca8, prio=0,
tag=0xffffffff8190d1da) at fs/zfs/rrwlock.c:181
#6  0xffffffff8181eb17 in dsl_pool_config_enter (dp=<optimized out>,
tag=0xffffffff8190d1da) at fs/zfs/dsl_pool.c:1338
#7  dsl_pool_hold (name=<optimized out>, tag=0xffffffff8190d1da,
dp=0xfffffe010e5a9848) at fs/zfs/dsl_pool.c:1310
#8  0xffffffff817f8991 in dmu_objset_hold (name=0xfffff80546b1a810
"testpool.1470/newvol", tag=0xffffffff8190d1da, osp=0xfffffe010e5a9888) at
fs/zfs/dmu_objset.c:626
#9  0xffffffff81823138 in dsl_prop_get (dsname=<unavailable>,
propname=0xffffffff8192fda6 "readonly", intsz=8, numints=1, buf=<optimized
out>, setpoint=0x0) at fs/zfs/dsl_prop.c:320
#10 dsl_prop_get_integer (ddname=<unavailable>, propname=0xffffffff8192fda6
"readonly", valuep=0xfffffe010e5a9908, setpoint=0x0) at fs/zfs/dsl_prop.c:343
#11 0xffffffff818dc3fc in zvol_first_open (zv=0xfffff80546b1a800) at
fs/zfs/zvol.c:880
#12 0xffffffff818dbdbb in zvol_open (pp=<optimized out>, count=<optimized out>,
flag=<optimized out>) at fs/zfs/zvol.c:1198
#13 zvol_geom_access (pp=0xfffff8062add8c00, acr=<optimized out>, acw=0,
ace=<optimized out>) at fs/zfs/zvol.c:2737
#14 0xffffffff804809f9 in g_access (cp=0xfffff804ef224000, dcr=<optimized out>,
dcw=0, dce=0) at sys/geom/geom_subr.c:977
#15 0xffffffff80486974 in g_part_taste (mp=0xffffffff80a7ebf0 <g_part_class>,
pp=0xfffff8062add8c00, flags=<optimized out>) at sys/geom/part/g_part.c:1982
#16 0xffffffff8048158a in g_new_provider_event (arg=0xfffff8062add8c00,
flag=<optimized out>) at sys/geom/geom_subr.c:557
#17 0xffffffff8047c3b4 in one_event () at sys/geom/geom_event.c:268
#18 g_run_events () at sys/geom/geom_event.c:291
#19 0xffffffff804ca144 in fork_exit (callout=0xffffffff8047e720
<g_event_procbody>, arg=0x0, frame=0xfffffe010e5a9ac0) at
sys/kern/kern_fork.c:1057
#20 <signal handler called>
========================================

The spa sync thread blocked on the spa_namespace_lock lock: 
------------------------------------------------------------
Thread 1774 (Thread 104418):
(struct thread *)0xfffff807bb1af580, tid 104418
   zfskern :: (struct proc *)0xfffff80005942528, pid 15

STACK:
#0  sched_switch (td=0xfffff807bb1af580, newtd=0xfffff80003ae7000,
flags=<optimized out>) at sys/kern/sched_ule.c:2112
#1  0xffffffff80513d6f in mi_switch (flags=260, newtd=0x0) at
sys/kern/kern_synch.c:439
#2  0xffffffff8055e04d in sleepq_switch (wchan=0xffffffff819aae98
<spa_namespace_lock>, pri=0) at sys/kern/subr_sleepqueue.c:613
#3  0xffffffff8055def3 in sleepq_wait (wchan=0xffffffff819aae98
<spa_namespace_lock>, pri=0) at sys/kern/subr_sleepqueue.c:692
#4  0xffffffff80511af6 in _sx_xlock_hard (sx=0xffffffff819aae98
<spa_namespace_lock>, x=<optimized out>, opts=<optimized out>, file=<optimized
out>, line=<optimized out>) at sys/kern/kern_sx.c:857
#5  0xffffffff805115e1 in _sx_xlock (sx=0xffffffff819aae98
<spa_namespace_lock>, opts=0, file=0xffffffff818ffa89 "fs/zfs/dsl_dataset.c",
line=3086) at sys/kern/kern_sx.c:328
#6  0xffffffff81815cc4 in dsl_dataset_promote_sync (arg=0xfffffe02597833d0,
tx=0xfffff80c74143000) at fs/zfs/dsl_dataset.c:3086
#7  0xffffffff8181f2d9 in dsl_sync_task_sync (dst=0xfffffe0259783308,
tx=0xfffff80c74143000) at fs/zfs/dsl_synctask.c:234
#8  0xffffffff8181d8db in dsl_pool_sync (dp=0xfffff80e53a8e800, txg=<optimized
out>) at fs/zfs/dsl_pool.c:819
#9  0xffffffff8184bfe3 in spa_sync (spa=<optimized out>, txg=<optimized out>)
at fs/zfs/spa.c:8124
#10 0xffffffff8185b206 in txg_sync_thread (arg=0xfffff80e53a8e800) at
fs/zfs/txg.c:543
#11 0xffffffff804ca144 in fork_exit (callout=0xffffffff8185af20
<txg_sync_thread>, arg=0xfffff80e53a8e800, frame=0xfffffe039cc11ac0) at
sys/kern/kern_fork.c:1057
#12 <signal handler called>

Pending on lock:
SX_LOCK: spa_namespace_lock, (struct sx *)0xffffffff819aae98
$5 = {
  lock_object = {
    lo_name = 0xffffffff8191dd6a,
    lo_flags = 0x22710000,
    lo_data = 0x0,
    lo_witness = 0x0
  },
  sx_lock = 0xfffff80003cdc584
}
LOCK HOLDER:
(struct thread *)0xfffff80003cdc580, tid 100163
   geom :: (struct proc *)0xfffff80003d6ba50, pid 13
========================================

Looking for the holder of the dp->dp_config_rwlock that the spa_namespace_lock
lock holder is blocked on:

>From the stack of Thread 1774:
set $dp = (dsl_pool_t *)0xfffff80e53a8e800

p $dp->dp_config_rwlock.rr_writer
$7 = (kthread_t *) 0xfffff807bb1af580

set $th = (struct thread *)0xfffff807bb1af580
p $th->td_tid
$8 = 104418

The spa sync thread is holding $dp->dp_config_rwlock.rr_writer for write
and it wants spa_namespace_lock while the cam "one_event" thread holds
spa_namespace_lock and wants the $dp->dp_config_rwlock.rr_writer for read.

-- 
You are receiving this mail because:
You are the assignee for the bug.


More information about the freebsd-fs mailing list