ARC size constantly shrinks, then ZFS slows down extremely

[Artem Belevich]

Your lockup is very similar (processes stuck sleeping on vmwait) to
what I had when arc_min was set too high. With Pawel's patch ZFS would
not give up any memory above arc_min.
Try bringing vfs.zfs.arc_min down.

--Artem



2009/10/5 Attila Nagy <bra at fsn.hu>:
> On 10/02/09 20:45, Pawel Jakub Dawidek wrote:
>>
>> On Fri, Oct 02, 2009 at 09:59:03AM +0200, Attila Nagy wrote:
>>
>>>
>>> Backing out this change from the 8-STABLE kernel:
>>>
>>> http://svn.freebsd.org/viewvc/base/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c?r1=191901&r2=191902
>>>
>>> makes it survive about half and hour of IMAP searching. Of course only
>>> time will tell whether this helps in the long run, but so far 10/10 tries
>>> succeeded to kill the machine with this method...
>>>
>>
>> Could you try this patch:
>>
>>        http://people.freebsd.org/~pjd/patches/arc.c.4.patch
>>
>
> Sure. But before that, a report with the above modification: the machine has
> survived some days, then started to behave strangely. Meaning I could ping
> it, I could log in to the IMAP service (running from ZFS), read some mails,
> but not all.
> I could not access it via ssh (which runs from UFS), but an already running
> top from a different session was alive. It showed:
> last pid: 11272;  load averages:  0.00,  0.00,  0.00    up 3+15:21:13
>  09:11:43
> 149 processes: 1 running, 143 sleeping, 1 zombie, 4 waiting
> CPU:  0.0% user,  0.0% nice,  0.2% system,  0.0% interrupt, 99.8% idle
> Mem: 234M Active, 197M Inact, 559M Wired, 111M Buf, 440K Free
> Swap: 4096M Total, 976K Used, 4095M Free
>
>  PID USERNAME  THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
> 78492 root        1  44    0  4700K  2156K CPU1    1   5:37  0.00% top
> 92343 root        1  44    0  4132K  1576K nanslp  1   4:12  0.00% gstat
> 13401 root        1  44    0  1528K   456K piperd  0   2:19  0.00%
> readproctitl
> 12679 root        1  44    0  3932K  1236K vmwait  1   2:12  0.00% zpool
> 35988    125      4  45    0 16892K  5968K sigwai  0   1:53  0.00%
> milter-greyl
> 25656 root        1  45    0  1536K   564K getblk  0   1:45  0.00% supervise
> 25798 root        1  44    0  1536K   564K vmwait  0   1:44  0.00% supervise
> 28406 root        1  44    0  1536K   544K vmwait  0   1:43  0.00% supervise
> 30226 root        1  44    0  1536K   544K vmwait  0   1:43  0.00% supervise
> 35401 root        1  44    0  1536K   544K vmwait  0   1:42  0.00% supervise
> 29203 root        1  44    0  1536K   544K vmwait  0   1:42  0.00% supervise
> 21629    389      6  44    0 91664K 41892K ucond   0   1:02  0.00% slapd
> 72283     60      1  44    0 80972K  1948K select  1   0:34  0.00% idled
> 98960 root        1  44    0  9396K  2544K select  1   0:32  0.00% sshd
> 1550 root        1  44    0  3340K   940K vmwait  1   0:32  0.00% syslogd
> 5463    125      1  44    0  6924K  2036K vmwait  0   0:27  0.00% qmgr
> 54193 root        1  44    0  9396K  2516K select  0   0:22  0.00% sshd
>
> I could not log into the console, it didn't even gave a "user name" filed
> after hitting enter. Strange.
>
> I will try the patch.
>
>
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>