zfs l2arc warmup

[Joar Jegleim]

Appreciate your input, and I've talked to our devs about that and
requested them to make some finit subset of the jpegs that rotate
every night.
Actually the whole web application is being rewritten, I won't have
anything like that until august at best, which certainly isn't bad .

When I get that kind of feature maybe I no longer need an l2arc to
cover the whole dataset.



On 27 March 2014 15:26, Bob Friesenhahn <bfriesen at simple.dallas.tx.us> wrote:
> On Thu, 27 Mar 2014, Joar Jegleim wrote:
>>
>> Is this how 'you' do it to warmup the l2arc, or am I missing something ?
>>
>> The thing is with this particular pool is that it serves somewhere
>> between 20 -> 30 million jpegs for a website. The front page of the
>> site will for every reload present a mosaic of about 36 jpegs, and the
>> jpegs are completely randomly fetched from the pool.
>> I don't know what jpegs will be fetched at any given time, so I'm
>> installing about 2TB of l2arc ( the pool is about 1.6TB today) and I
>> want the whole pool to be available from the l2arc .
>
>
> Your usage pattern is the opposite of what the ARC is supposed to do. The
> ARC is supposed to keep most-often accessed data in memory (or retired to
> L2ARC) based on access patterns.
>
> It does not seem necessary for your mosaic to be truely random across
> 20 -> 30 million jpegs.  Random across 1000 jpegs which are circulated
> in time would produce a similar effect.
>
> The application building your web page mosiac can manage which files will be
> included in the mosaic and achieve the same effect as a huge cache by always
> building the mosiac from a known subset of files. The 1000 jpegs used for
> the mosaics can be cycled over time from a random selection, with old ones
> being removed.  This approach assures that in-memory caching is effective
> since the same files will be requested many times by many clients.
>
> Changing the problem from an OS-oriented one to an application-oriented one
> (better algorithm) gives you more control and better efficiency.
>
> Bob
> --
> Bob Friesenhahn
> bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
> GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/



-- 
----------------------
Joar Jegleim
Homepage: http://cosmicb.no
Linkedin: http://no.linkedin.com/in/joarjegleim
fb: http://www.facebook.com/joar.jegleim
AKA: CosmicB @Freenode

----------------------