zfs l2arc warmup

[Bob Friesenhahn]

On Thu, 27 Mar 2014, Joar Jegleim wrote:
> Is this how 'you' do it to warmup the l2arc, or am I missing something ?
>
> The thing is with this particular pool is that it serves somewhere
> between 20 -> 30 million jpegs for a website. The front page of the
> site will for every reload present a mosaic of about 36 jpegs, and the
> jpegs are completely randomly fetched from the pool.
> I don't know what jpegs will be fetched at any given time, so I'm
> installing about 2TB of l2arc ( the pool is about 1.6TB today) and I
> want the whole pool to be available from the l2arc .

Your usage pattern is the opposite of what the ARC is supposed to do. 
The ARC is supposed to keep most-often accessed data in memory (or 
retired to L2ARC) based on access patterns.

It does not seem necessary for your mosaic to be truely random across
20 -> 30 million jpegs.  Random across 1000 jpegs which are circulated
in time would produce a similar effect.

The application building your web page mosiac can manage which files 
will be included in the mosaic and achieve the same effect as a huge 
cache by always building the mosiac from a known subset of files. 
The 1000 jpegs used for the mosaics can be cycled over time from a 
random selection, with old ones being removed.  This approach assures 
that in-memory caching is effective since the same files will be 
requested many times by many clients.

Changing the problem from an OS-oriented one to an 
application-oriented one (better algorithm) gives you more control and 
better efficiency.

Bob
-- 
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/