l2arc_feed_thread cpu utlization

[Andriy Gapon]

on 14/02/2014 13:52 Andriy Gapon said the following:
> on 19/12/2013 13:30 Andriy Gapon said the following:
>>
>> This is just a heads up, no patch yet.
>>
>> l2arc_feed_thread periodically wakes up and scans certain amount of ARC buffers
>> and writes eligible buffers to a cache device.
>> Number of scanned buffers is limited by a threshold on the amount of data in the
>> buffers seen.  The threshold is applied on a per buffer list basis.  In upstream
>> there are 4 relevant lists: (data, metadata) X (MFU, MRU).  In FreeBSD each of
>> the lists was subdivided into 16 lists.  This was done to reduce contention on
>> the locks that protect the lists.  But as a side effect l2arc_feed_thread can
>> scan 16 times more data (~ buffers).
>>
>> So, if you have a rather large ARC and L2ARC and your buffers tend to be
>> sufficiently small, then you could observe l2arc_feed_thread burning a
>> noticeable amount of CPU.  On some of our systems I observed it using up to 40%
>> of a single core.  Scaling back the threshold by factor of 16 makes CPU
>> utilization go down by the same factor.
>>
>> I plan to commit this change to FreeBSD ZFS code.
>> Any comments are welcome.
> 
> Here is what I have in mind:
> https://github.com/avg-I/freebsd/compare/wip;hc;l2arc_feed_thread_scan_rate
> 
> The calculations in the macro look somewhat ugly, but they should be correct :-)
> 

Looks like the patch did more than I wanted.  Not only it limited how much data
is scanned per list, but it also limited how much data was written in total.
So, this new patch should be better:
https://github.com/avg-I/freebsd/compare/master...review;l2arc-feed-thread-scan-size.diff

-- 
Andriy Gapon