ZFS memory management

[Nikolay Denev]

Hello list,

I have the following question : I have several machines with 196G of RAM that are using
RELENG_9 with ZFS, and are running a very memory intensive java applications - ElasticSearch
The machines are without swap configured and have "vm.swap_enabled=0" in /etc/sysctl.conf.
The ElasticSearch processes are using mlockall(2) to pin down their memory (configured at 40G).
And at this point I thought that there would be no problems, but from time to time, when the machine grows it's 
ARC memory and there are some other running processes like nginx with passenger and uwsgi the ElasticSearch
process would get killed by the kernel OOM killer with reason "no swap space available"

Of course, I've now tuned down arc_max in /boot/loader.conf, but isn't this supposed to work automatically? Like
ZFS releasing some memory when there is a pressure, instead of the OOM killer going postal? (at the moment when
the process was killed the ZFS ARC was 132G).

I understand that this might be problematic as AFAIK ZFS releases memory asynchronously when the arc_reclaim_thread() is run,
which might take some time to be scheduled and complete.

Cheers,
Nikolay