kern.geom.journal.stats.low_mem refers to what?

Luke Dean LukeD at
Thu Mar 31 02:07:26 UTC 2011

> I've recently set up gjournal on top of gmirror on FreeBSD 8.2.  I understand 
> that this setup has a lot of redundant writing.  It is working, but I'm not 
> sure I've set it up as efficiently as I should.
> During prolonged writes, such as copying large files to the file system 
> across the network or producing very large logfiles, the "low_mem" and 
> "skipped_bytes" statisics rise rapidly and the system becomes less 
> responsive.  "top" always reports free memory, so I don't think that's the 
> issue.  "journal_full" and "wait_for_copy" have never occurred.  Here's a 
> sample of what happens after a couple hours of intense writing...
> kern.geom.journal.stats.low_mem: 5379
> kern.geom.journal.stats.journal_full: 0
> kern.geom.journal.stats.wait_for_copy: 0
> kern.geom.journal.stats.switches: 7543
> kern.geom.journal.stats.combined_ios: 265318
> kern.geom.journal.stats.skipped_bytes: 935712768
> "low_mem" sounds like a bad thing.  What could I do to remedy that?  Did I 
> make the journal too small?  The stats say that "journal_full" has never 
> happened, so maybe not?  Is there a setting I should tweak?
> The Handbook says 1GB is good enough most of the time, but it also says that 
> 3x the amount of physical memory is a good size as well.  I compromised 
> between the two and made an 8GB journal for this system that has ~4GB of 
> memory.

Replying to myself for posterity...  I think I figured it out.

I believe "low_mem" means its running out of gjournal cache, defined by 
kern.geom.journal.cache.limit, which is initially set by 
kern.geom.journal.cache.divisor, which is initially 2, which means half of 
the kernel memory, which is vm.kmem_size, which is 330MB on my i386 8.2 
system by default.  I don't know where this default comes from, but it 
seems common.  This gives me about 167MB of cache for journalling.

During big file transfers, I can watch the "journal_data" statistic in 
"vmstat -m".  It climbs close to the 167MB cache size.  That's when I see 
kern.geom.journal.stats.low_mem tick over and the stutter happens.  So I 
think I understand what I'm seeing now.

I could try increasing vm.kmem_size and possibly the KMEM_SIZE option 
that's compiled into my kernel as described in the ZFS tuning documents, 
since it's a similar issue.  This would allow the kernel to have more 
memory and thus allow gjournal to use more memory for caching.  If I'm 
consistently filling the cache faster than I can empty it, making the 
cache bigger probably won't help.  This journal is on top of a mirror, so 
I'm always expecting it to read faster than it writes...  Copying large 
files locally might always max out the buffer?  No harm in trying, I 

Or I could just live with it.  It's doing what I told it to do.

More information about the freebsd-questions mailing list