zfs problem with processes stuck in state zfs and zilog-
Kai Gallasch
gallasch at free.de
Tue Jan 26 12:06:12 UTC 2010
Hi.
Some mail services (qmail, dovecot) on one of my freebsd jails became
unavailable. The jail is using zfs as a filesystem. After logging in I
found that I could not kill processes related to the unavailable
services running inside the jail and that the jail could not be
restartet.
top revealed the following:
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
8320 root 1 44 0 4772K 1088K zilog- 1 0:58 0.00% multilog
8336 qmaill 1 44 0 4772K 1088K zilog- 6 0:21 0.00% multilog
8464 mysql 2 44 0 108M 70356K STOP 7 0:21 0.00% mysqld
8292 spamd 1 44 0 4772K 1088K zilog- 7 0:18 0.00% multilog
8295 nobody 1 44 0 4772K 1088K zilog- 2 0:10 0.00% multilog
8298 qmaill 1 44 0 4772K 1088K zilog- 3 0:01 0.00% multilog
8294 root 1 44 0 4772K 1088K zilog- 4 0:01 0.00% multilog
8304 qmaill 1 44 0 4772K 1088K zilog- 1 0:01 0.00% multilog
34278 www 1 63 0 29852K 14828K zfs 4 0:00 0.00% perl
34368 www 1 76 0 29852K 14828K zfs 4 0:00 0.00% perl
8319 root 1 44 0 4772K 1088K zilog- 2 0:00 0.00% multilog
4311 qmaild 1 59 0 2792K 1040K zilog- 1 0:00 0.00% qmail-queue
32203 qmaild 1 60 0 2792K 1040K zilog- 0 0:00 0.00% qmail-queue
16225 qmaild 1 59 0 2792K 1040K zilog- 4 0:00 0.00% qmail-queue
32254 qmaild 1 60 0 2792K 1040K zilog- 0 0:00 0.00% qmail-queue
16545 qmaild 1 57 0 2792K 1040K zilog- 4 0:00 0.00% qmail-queue
4521 qmaild 1 56 0 2792K 1040K zilog- 5 0:00 0.00% qmail-queue
8327 root 1 44 0 4772K 1088K zilog- 6 0:00 0.00% multilog
36733 qmaild 1 52 0 2792K 1040K zilog- 4 0:00 0.00% qmail-queue
47903 root 1 44 0 10284K 3412K pause 0 0:00 0.00% tcsh
35517 qmaild 1 63 0 2792K 1040K zilog- 4 0:00 0.00% qmail-queue
35492 qmaild 1 63 0 2792K 1040K zilog- 0 0:00 0.00% qmail-queue
62826 qmaild 1 53 0 2792K 1040K zilog- 4 0:00 0.00% qmail-queue
69033 qmaild 1 53 0 2792K 1040K zilog- 0 0:00 0.00% qmail-queue
66010 qmaild 1 50 0 2792K 1040K zilog- 4 0:00 0.00% qmail-queue
40947 qmaild 1 50 0 2792K 1040K zilog- 0 0:00 0.00% qmail-queue
27726 qmaild 1 51 0 2792K 1040K zilog- 0 0:00 0.00% qmail-queue
35520 qmaild 1 53 0 2792K 1040K zilog- 6 0:00 0.00% qmail-queue
30639 root 1 44 0 12164K 3420K zfs 1 0:00 0.00% perl
23538 root 1 44 0 12164K 3420K zfs 1 0:00 0.00% perl
4834 root 1 76 0 12164K 3420K zfs 1 0:00 0.00% perl
26410 root 1 47 0 12164K 3420K zfs 0 0:00 0.00% perl
97136 root 1 76 0 12164K 3420K zfs 4 0:00 0.00% perl
9290 root 1 44 0 12164K 3420K zfs 1 0:00 0.00% perl
18127 root 1 44 0 12164K 3420K zfs 5 0:00 0.00% perl
12553 root 1 44 0 12164K 3420K zfs 4 0:00 0.00% perl
21574 root 1 44 0 12164K 3420K zfs 4 0:00 0.00% perl
2323 root 1 44 0 12164K 3420K zfs 0 0:00 0.00% perl
68707 root 1 45 0 12164K 3420K zfs 2 0:00 0.00% perl
9294 root 1 44 0 12164K 3420K zfs 1 0:00 0.00% perl
40393 root 1 44 0 12164K 3420K zfs 1 0:00 0.00% perl
41985 root 1 44 0 12164K 3420K zfs 0 0:00 0.00% perl
21672 root 1 44 0 12164K 3420K zfs 0 0:00 0.00% perl
10420 root 1 44 0 12164K 3420K zfs 4 0:00 0.00% perl
6965 root 1 44 0 12164K 3420K zfs 6 0:00 0.00% perl
14132 root 1 44 0 12164K 3420K zfs 1 0:00 0.00% perl
14867 root 1 44 0 12164K 3420K zfs 4 0:00 0.00% perl
77145 root 1 44 0 12164K 3420K zfs 4 0:00 0.00% perl
16534 root 1 44 0 12164K 3420K zfs 0 0:00 0.00% perl
25130 root 1 44 0 12164K 3420K zfs 0 0:00 0.00% perl
63026 root 1 44 0 12164K 3420K zfs 4 0:00 0.00% perl
44663 root 1 44 0 12164K 3420K zfs 4 0:00 0.00% perl
28139 root 1 44 0 12164K 3420K zfs 0 0:00 0.00% perl
34440 root 1 44 0 12164K 3420K zfs 4 0:00 0.00% perl
35619 qmaild 1 46 0 2792K 1040K zilog- 0 0:00 0.00% qmail-queue
26146 root 1 44 0 12164K 3420K zfs 0 0:00 0.00% perl
Because the server is in active use, there was no time
for me to look into this further - and no trace of the problem in syslog.
Any suggestions how to debug this problem and find out what's wrong here?
Does the state zilog- give any hint?
BTW: FreeBSD 8.0-STABLE-amd64 / zpool version 13
--Kai
--
You had mail, but the super-user read it,
laughed and deleted it!
More information about the freebsd-fs
mailing list