zfs l2arc warmup

[Ronald Klop]

On Thu, 27 Mar 2014 08:50:06 +0100, Joar Jegleim <joar.jegleim at gmail.com>  
wrote:

> Hi list !
>
> I struggling to get a clear understanding of how the l2arc get warm (  
> zfs).
> It's a FreeBSD 9.2-RELEASE server.
>
> From various forum I've come up with this which I have in my  
> /boot/loader.conf
> # L2ARC tuning
> # Maximum number of bytes written to l2arc per feed
> # 8MB (actuall=vfs.zfs.l2arc_write_max*(1000 /  
> vfs.zfs.l2arc_feed_min_ms))
> # so 8MB every 200ms = 40MB/s
> vfs.zfs.l2arc_write_max=8388608
> # Mostly only relevant at the first few hours after boot
> # write_boost, speed to fill l2arc until it is filled (after boot)
> # 70MB, same rule applys, multiply by 5 = 350MB/s
> vfs.zfs.l2arc_write_boost=73400320
> # Not sure
> vfs.zfs.l2arc_headroom=2
> # l2arc feeding period
> vfs.zfs.l2arc_feed_secs=1
> # minimum l2arc feeding period
> vfs.zfs.l2arc_feed_min_ms=200
> # control whether streaming data is cached or not
> vfs.zfs.l2arc_noprefetch=1
> # control whether feed_min_ms is used or not
> vfs.zfs.l2arc_feed_again=1
> # no read and write at the same time
> vfs.zfs.l2arc_norw=1
>
> But what I really wonder is how does the l2arc get warmed up ?
> I'm thinking of 2 scenarios:
>
> a.: when arc is full, stuff that evict from arc is put over in l2arc,
> that means that files in the fs that are never accessed will never end
> up in l2arc, right ?
>
> b.: zfs run through fs in the background and fill up the l2arc for any
> file, regardless if it has been accessed or not ( this is the
> 'feature' I'd like )
>
> I suspect scenario a is what really happens, and if so, how does
> people warmup the l2arc manually (?)
> I figured that if I rsync everything from the pool I want to be
> cache'ed, it will fill up the l2arc for me, which I'm doing right now.
> But it takes 3-4 days to rsync the whole pool .
>
> Is this how 'you' do it to warmup the l2arc, or am I missing something ?
>
> The thing is with this particular pool is that it serves somewhere
> between 20 -> 30 million jpegs for a website. The front page of the
> site will for every reload present a mosaic of about 36 jpegs, and the
> jpegs are completely randomly fetched from the pool.
> I don't know what jpegs will be fetched at any given time, so I'm
> installing about 2TB of l2arc ( the pool is about 1.6TB today) and I
> want the whole pool to be available from the l2arc .
>
>
> Any input on my 'rsync solution' to warmup the l2arc is much appreciated  
> :)


2TB of l2arc?
Why don't you put your data on SSD's, get rid of the l2arc and buy some  
extra RAM instead.
Than you don't need any warm-up.

For future questions, please provide more details about your setup. What  
are disks, what ssds, how much RAM. How is your pool configured? Mirror,  
raidz, ... Things like that.

Ronald.