kmem_map too small with ZFS and 8.2-RELEASE
Jeremy Chadwick
freebsd at jdc.parodius.com
Fri Mar 4 14:38:13 UTC 2011
On Fri, Mar 04, 2011 at 08:23:29AM -0500, Adam McDougall wrote:
> On 03/04/11 04:48, Mickaël Canévet wrote:
> >>I'd use vm.kmem_size="32G" (i.e. twice your RAM) and that's it.
> >
> >Should I also increase vfs.zfs.arc_max ?
> >
> >Do you have any idea why the kernel panicked at only 8GB allocated ?
> >
> >Thank you
>
> I believe ARC allocations in kmem can become fragmented, so when it
> is searching for a place to store a new contiguous segment of
> memory, the remaining fragmented free spaces may all be too small.
> I also set
> vm.kmem_size to about twice the amount of ram to help it avoid this
> issue. I suspect if kmem is badly fragmented then performance of
> ZFS can downgrade, so that is another reason to keep kmem bigger.
My findings on 8.2-RELEASE indicate that doing this results in very
unexpected behaviour regarding the ARC maximum. As such, I cannot
recommend this model.
For example, on an amd64 system with 8GB physical RAM and these two
settings in /boot/loader.conf:
vm.kmem_size="8192M"
vfs.zfs.arc_max="6144M"
kstat.zfs.misc.arcstats.size tops out at around 6240986896, with Wired
in top(1) showing ~6.3GB. This is expected behaviour and fits (I
think) what people expect.
However, on the exact same system with these two settings:
vm.kmem_size="16384M"
vfs.zfs.arc_max="6144M"
The above ARC numbers are exactly *half* that amount. This is easily
reproducible.
Can someone 1) justify the "2x the amount of RAM for vm.kmem_size"
setting, and 2) explain in detail the above behaviour?
--
| Jeremy Chadwick jdc at parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, USA |
| Making life hard for others since 1977. PGP 4BD6C0CB |
More information about the freebsd-fs
mailing list