ARC "pressured out", how to control/stabilize ? (reformatted to text/plain)

[Vladimir Sharun]

Dear Andriy and FreeBSD community,

> I am not sure if the buffers are leaked somehow or if they are actually in use.
> It's one of the very few places where data buffers are allocated without
> charging ARC.  In all other places it's quite easy to match allocations and
> deallocations.  But in L2ARC it is not obvious that all buffers get freed or
> when that happens.

After one week under load I think we figure out the cause: it's L2ARC. 
Here's the top's header for 7d17h of the runtime:

last pid: 46409;  load averages:  0.37,  0.62,  0.70 up 7+17:14:01  07:24:10
173 processes: 1 running, 171 sleeping, 1 zombie
CPU:  2.0% user,  0.0% nice,  3.5% system,  0.4% interrupt, 94.2% idle
Mem: 8714M Active, 14G Inact, 96G Wired, 1929M Cache, 3309M Buf, 3542M Free
ARC: 85G Total, 2558M MFU, 77G MRU, 28M Anon, 1446M Header, 4802M Other

ARC related tunables:

vm.kmem_size="110G"
vfs.zfs.arc_max="90G"
vfs.zfs.arc_min="42G"

For more than 7 days of hard runtime the picture clearly shows: 
Wired minus ARC = 11..12Gb, ARC grow and shrinks in 80-87Gb range and the
system runs just fine.

So what shall we do with L2ARC leakage ?