ARC size limit

Tue Nov 4 16:48:19 UTC 2014

Thanks Steven. We are running 10.1rc3. Is the best way to get this patch to just compile 10-stable kernel ? We are new to running FreeBSD and hence the newbie question.

Thanks,
George

> On Nov 4, 2014, at 1:06 AM, Steven Hartland <killing at multiplay.co.uk> wrote:
> 
> You need https://svnweb.freebsd.org/base?view=revision&revision=272875
> On 04/11/2014 06:29, George Kola wrote:
>> Hi All,
>>        This is my first post to freebsd-stable fresh of Meet BSD California 2014.  We are switching our entire production to FreeBSD. Our storage servers have  256 GB of RAM , 4 TB of SSD and 40 TB of spinning disks. We are running ZFS root and the SSD is configured as L2ARC.  We are running FreeBSD 10.1 RC3
>>        I am finding that on all our machines, ARC is somehow limited to < 64 GB of memory and we have a huge inactive memory (180 G). The surprising thing is that ARC seems to have almost the same limit  (< 64 GB) on all of our storage boxes and ARC is not growing even though L2ARC hit shows that there is advantage in growing ARC.
>>        Any help/pointers is appreciated.
>>        What I am trying to do is to tune ZFS for our workload. We are hoping that we get a high hit rate.
>>        Thanks to Justin Gibbs and Allan Jude for initial pointers and help. They suggested posting to the mailing list to get further help.
>> 
>>        I have pasted top output and zfs-stats output below and yes UMA is enabled.
>> 
>> Thanks,
>> George
>>        
>> 
>> top
>> last pid: 27458;  load averages:  3.30,  5.42,  5.34                                                                                                up 6+09:59:30  05:38:49
>> 71 processes:  1 running, 70 sleeping
>> CPU:  4.2% user,  0.0% nice,  4.6% system,  0.2% interrupt, 90.9% idle
>> Mem: 11G Active, 181G Inact, 52G Wired, 1368M Cache, 4266M Free
>> ARC: 47G Total, 1555M MFU, 41G MRU, 35M Anon, 3984M Header, 709M Other
>> Swap: 64G Total, 2874M Used, 61G Free, 4% Inuse
>> 
>> 
>> 
>> sysctl vfs.zfs.zio.use_uma
>> vfs.zfs.zio.use_uma: 1
>> 
>> 
>> 
>> 
>> zfs-mon -a output
>> 
>> ZFS real-time cache activity monitor
>> Seconds elapsed:  62
>> 
>> Cache hits and misses:
>>                                   1s    10s    60s    tot
>>                      ARC hits:   124    126    103    101
>>                    ARC misses:    35     46     29     28
>>          ARC demand data hits:    55     90     61     61
>>        ARC demand data misses:    20     32     18     17
>>      ARC demand metadata hits:    69     36     42     40
>>    ARC demand metadata misses:     9     13     10      9
>>        ARC prefetch data hits:     0      0      0      0
>>      ARC prefetch data misses:     6      1      1      1
>>    ARC prefetch metadata hits:     0      0      0      0
>>  ARC prefetch metadata misses:     0      0      0      0
>>                    L2ARC hits:    16     28     14     14
>>                  L2ARC misses:    19     18     15     14
>>                   ZFETCH hits:   592   2842   2098   2047
>>                 ZFETCH misses:   308   1326    507    494
>> 
>> Cache efficiency percentage:
>>                           10s    60s    tot
>>                   ARC:  73.26  78.03  78.29
>>       ARC demand data:  73.77  77.22  78.21
>>   ARC demand metadata:  73.47  80.77  81.63
>>     ARC prefetch data:   0.00   0.00   0.00
>> ARC prefetch metadata:   0.00   0.00   0.00
>>                 L2ARC:  60.87  48.28  50.00
>>                ZFETCH:  68.19  80.54  80.56
>> 
>> 
>> 
>> 
>> zfs-stats -a output
>> 
>> ZFS real-time cache activity monitor
>> Seconds elapsed:  62
>> 
>> Cache hits and misses:
>>                                   1s    10s    60s    tot
>>                      ARC hits:   124    126    103    101
>>                    ARC misses:    35     46     29     28
>>          ARC demand data hits:    55     90     61     61
>>        ARC demand data misses:    20     32     18     17
>>      ARC demand metadata hits:    69     36     42     40
>>    ARC demand metadata misses:     9     13     10      9
>>        ARC prefetch data hits:     0      0      0      0
>>      ARC prefetch data misses:     6      1      1      1
>>    ARC prefetch metadata hits:     0      0      0      0
>>  ARC prefetch metadata misses:     0      0      0      0
>>                    L2ARC hits:    16     28     14     14
>>                  L2ARC misses:    19     18     15     14
>>                   ZFETCH hits:   592   2842   2098   2047
>>                 ZFETCH misses:   308   1326    507    494
>> 
>> Cache efficiency percentage:
>>                           10s    60s    tot
>>                   ARC:  73.26  78.03  78.29
>>       ARC demand data:  73.77  77.22  78.21
>>   ARC demand metadata:  73.47  80.77  81.63
>>     ARC prefetch data:   0.00   0.00   0.00
>> ARC prefetch metadata:   0.00   0.00   0.00
>>                 L2ARC:  60.87  48.28  50.00
>>                ZFETCH:  68.19  80.54  80.56
>> 
>> 
>> 
>> _______________________________________________
>> freebsd-stable at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
>> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
> 
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"