Memory leak in ZFS?

Bartosz Stec admin at
Tue Feb 8 10:48:20 UTC 2011

W dniu 2011-02-08 11:27, Jeremy Chadwick pisze:
> On Tue, Feb 08, 2011 at 10:24:11AM +0100, Bartosz Stec wrote:
>> W dniu 2011-02-07 22:37, Emil Muratov pisze:
>>>> For the past few weeks, I noticed that the amount of memory
>>>> reported in top
>>>> (sum of active, inact, wired, cache buf and free) keeps
>>>> decreasing as the
>>>> uptime increases. I can't pinpoint to when I first noticed this,
>>>> as I have
>>>> updated the system a few times just in case this has been fixed.
>>> Yes, I have the same issue on my home file storage. My system is
>>> 8.1 amd64, 2G ram, zfs on root raidz with 4x1,5T drives.
>>> After updating to stable a couple of days ago I noticed that the
>>> system leaks memory very fast. Checking here and there I found
>>> that the issue concerns sendfile (yep, again!).
>>> How to reproduce:
>>> Configure samba with aio and sendfile (mine is version 3.5.6)
>>> smb.conf
>>> [global]
>>> use sendfile=true
>>> aio read size = 16384
>>> Download a couple of large samba shared files (8-10 gigs).
>>> While downloading files I can see that memory decreazes to nowhere
>>> very-very fast, several MBs per second! First it drains free mem,
>>> than active and inactive, than comes wired until the whole system
>>> commits suicide suffocating itself to the death.
>>> The only way to free memory is to reboot the system. I can't
>>> unload zfs module like PJD suggested to do, 'cause my root is on
>>> zfs :(
>>> I'll try to make a bootable flash and move root to the flash to
>>> try to unload module and what will happen.
>>> Everything was OK in stable before the new year, sendfile used to
>>> pump free and wired memory to inactive than slowly reclaiming it
>>> back. But it seems something was changed after NY holydays?
>> I'm glad someone else finally picked that problem, so there's
>> appareantly no memory-eating ghost in my machine ;)
>> Here's my thread on stable list about this issue:
>> And in fact, PC reported in thread above is also SAMBA server with
>> aio/sendfile enabled and ZFS.
>> I would be happy testing some patches if necessary, because until
>> now I need to monitor memory and reboot this server before it dies.
> The source and build date of your kernel will matter greatly here.
> I can't speak about the memory utilisation aspect, but I tend to disable
> sendfile everywhere possible when ZFS is in use on a system.  The reason
> is based on something I and another user experienced back in October
> 2010 pertaining to sendfile() on ZFS locking up processes (making them
> unkillable).  See here[1] for details; this problem has since been
> fixed[2] (look for commits around October).  You'll also find some
> commits that went through in November pertaining to ZFS and sendfile.
> This is why I said the date of your kernel/sources matters.  :-)

I tried rebuild since original thread, hoping that problem is fixed 
already, so now it's very fresh: 8.2-PRERELEASE #18: Sun Feb  6 03:04:47 
CET 2011.
Problem is still here:

Mem: 37M Active, 78M Inact, 1154M Wired, 64M Cache, 199M Buf, 40M Free
About 1373MB instead of 2GB, and it's not even 2 days of uptime.

> The issue I referenced in [1] is not related to memory utilisation, but
> does indicate use of sendfile with ZFS may be a bad idea (by this I
> mean, there may be aspects of its implementation when mixed with ZFS
> that have been overlooked).
> Simple test: if you disable use of sendfile (but not AIO) in Samba, does
> the problem go away?
I've just disabled sendfile in smb.conf and I'll report in about 2 days, 
after reboot which I will perform tonight.
I hope it won't hit samba performance too much ;)

Bartosz Stec

More information about the freebsd-fs mailing list