ZFS L2arc 16.0E size

[Frank de Bot (lists)]

Slawa Olhovchenkov wrote:
> On Mon, Feb 23, 2015 at 12:21:35AM +0300, Slawa Olhovchenkov wrote:
> 
>> On Mon, Feb 16, 2015 at 12:52:17AM +0000, Steven Hartland wrote:
>>
>>> IIRC this was fixed by r273060, if your remove your cache device and 
>>> then add it back I think you should be good.
>>
>> r273060 will be reverted.
>> I am use stable (r276179) and still have (partialy) this problem:
>>
>> cache                                       -      -      -      -      -      -
>>   ada0                                   477G   477G  16.0E         -     0%   100%
>>   ada1                                   477G   477G  16.9M         -     0%    99%
>>   da2                                    477G   477G  25.0M         -     0%    99%
>>   da3                                    477G   477G  14.6M         -     0%    99%
>>   da4                                    477G   477G  30.6M         -     0%    99%
>>   da5                                    477G   477G  15.0M         -     0%    99%
>>   da6                                    477G   477G   298K         -     0%    99%
>>   da7                                    477G   477G  18.2M         -     0%    99%
> 
> same server, some time late:
> cache                                       -      -      -      -      -      -
>   ada0                                   477G   477G  16.8M         -     0%    99%
>   ada1                                   477G   477G  32.1M         -     0%    99%
>   da2                                    477G   477G  5.70M         -     0%    99%
>   da3                                    477G   477G  16.0E         -     0%   100%
>   da4                                    477G   477G  26.1M         -     0%    99%
>   da5                                    477G   477G  26.6M         -     0%    99%
>   da6                                    477G   477G  16.0E         -     0%   100%
>   da7                                    477G   477G  16.0E         -     0%   100%
> 
> hmm, -p don't affect -v values.

Tonight I had run some tests:
 - Disabling compression (by changing ZIO_COMPRESS_LZ4 to
ZIO_COMPRESS_OFF in sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c
in l2arc_compress_buf
 - disabling trim by vfs.zfs.trim.enabled=0 in loader.conf
 - trying to prevent io queues, but I don't think this was good. I  was
thinking there maybe is a possibility that between filling a writebuffer
for a vdev the size of l2arc was checked, but didn't take in account
that not all data is there (I really don't have a clue if  this makes
any sense)

None of these test were successful, every time free space is 16.0E and
used space keeps on growing and growing.

Slawa, I do notice that first ada0 was at 16.0E, but at your second post
not anymore.