ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks

[Artem Belevich]

On Fri, Jan 7, 2011 at 3:16 AM, Matthew D. Fuller
<fullermd at over-yonder.net> wrote:
> On Thu, Jan 06, 2011 at 03:45:04PM +0200 I heard the voice of
> Daniel Kalchev, and lo! it spake thus:
>>
>> You should also know that having large L2ARC requires that you also
>> have larger ARC, because there are data pointers in the ARC that
>> point to the L2ARC data. Someone will do good to the community to
>> publish some reasonable estimates of the memory needs, so that
>> people do not end up with large but unusable L2ARC setups.
>
> Estimates I've read in the past are that L2ARC consumes ARC space at
> around 1-2%.

Each record in L2ARC takes about 250 bytes in ARC. If I understand it
correctly, not all records are 128K which is default record size on
ZFS. If you end up with a lot of small records (for instance, if you
have a lot of small files or due to a lot of synchronous writes or if
record size is set to a lower value) then you could potentially end up
with much higher ARC requirements.

So, 1-2% seems to be a reasonable estimate assuming that ZFS deals
with ~10K-20K records most of the time. If you mostly store large
files your ratio would probably be much better.

One way to get specific ratio for *your* pool would be to collect
record size statistics from your pool using "zdb -L -b <pool>" and
then calculate L2ARC:ARC ratio based on average record size. I'm not
sure, though whether L2ARC stores records in compressed or
uncompressed form.

--Artem