8.1-STABLE: zfs and sendfile: problem still exists

[Ronald Klop]

On Sun, 31 Oct 2010 10:02:44 +0100, Alexander Zagrebin <alexz at visp.ru>  
wrote:

>> >> I apologize for my haste, it should have been VM_ALLOC_WIRED.
>> >
>> > Ok, applied and tested under some load(~1200 active
>> connections, outgoing
>> > ~80MB/s). Patch work as expected and i has noted no side
>> effects.  Just one
>> > question - should grow Active memory counter, if some pages
>> is "hot"(during
>> > multiple sendfile on one file)?
>>
>> Pages used by sendfile are marked as Inactive for faster
>> reclamation on demand.
>
> I have a question.
> When we transfer a file via sendfile, then current code allocates
> a memory, marked inactive. For example, if the file has size 100 MB,
> then 100 MB of memory will be allocated.
> If we have to transfer this file again later, then this memory will used
> as cache, and no disk io will be required.
> The memory will be freed if file will be deleted or operating system
> will need an additional memory.
> I have correctly understood?
> If it so, the i continue...
> Such behaviour is good if we have files with relatively small size.
> Suppose we have to transfer file with large size (for example, greater
> than amount of physical memory).
> While transfering, the inactive memory will grow, pressing the ARC.
> When size of the ARC will fall to its minimum (vfs.zfs.arc_min), then
> inactive memory will be reused.
> So, when transfer is complete, we have:
> 1. No free memory
> 2. Size of the ARC has minimal size (it is bad)
> 3. Inactive memory contains the _tail_ of the file only (it is bad too)
> Now if we have to transfer this file again, then
> 1. there is no (or few) file's data in ARC (ARC too small)
> 2. The inactive memory doesn't contain a head part of the file
> So the file's data will read from a disk again and again...
> Also i've noticed that inactive memory frees relatively slowly,
> so if there is a frequent access to large files, then system will run
> at very unoptimal conditions.
> It's imho...
> Can you comment this?
>

Add more RAM?