ZFS eats all memory
Dr. Nikolaus Klepp
dr.klepp at gmx.at
Fri Apr 26 06:41:19 UTC 2019
I have an interesting problem with FreeBSD FreeBSD 12.0-RELEASE-p3 GENERIC. Server is running ZFS mirror with 2 disks, 1TB each, 8GB RAM, 16GB Swap. I had reduced ARC to 4G, but everything ele is default:
All applications under full load use ~ 500MB Ram.
20 days later all memery was used by the kernel, cron could not spawn any subprosses, login and everything despite ZFS was gone - even USB was gone (I tried a USB keyboard, no lights went on). I had to reboot the server and everything went back to normal. The only hint in the logfiles was that cron could not spawn any process.
So I reduced ARC to 2GB:
Warned by the previuse behaviour I closely watched RAM usage. After ~ 7 Days the kernel alread used 7GB RAM, all wired, no swap used. Now I reduced kernel memory and rebooted the machine:
That was yesterday. Now I'm watching wired RAM usage go up again, but I think it does not go above 4GB. top says:
Mem: 98M Active, 190M Inact, 3947M Wired, 3563M Free
ARC: 1968M Total, 979M MFU, 512M MRU, 32K Anon, 85M Header, 392M Other
1240M Compressed, 3755M Uncompressed, 3.03:1 Ratio
Swap: 16G Total, 16G Free
I am quite surprised by this behaviour. I have not seen this happen on 11.2, but that server had 256GB RAM, not 8GB.
Is this expected to happen or should I be aware of some other pitfalls? E.g. can I be sure that the system will run longer than 20 days without lockups?
Please do not email me anything that you are not comfortable also sharing with the NSA, CIA ...
More information about the freebsd-questions