svn commit: r291460 - head/sys/kern

[Kirk McKusick]

> To: Kirk McKusick <mckusick at FreeBSD.org>, src-committers at freebsd.org,
>         svn-src-all at freebsd.org, svn-src-head at freebsd.org
> From: Kubilay Kocak <koobs at FreeBSD.org>
> Subject: Re: svn commit: r291460 - head/sys/kern
> Date: Mon, 30 Nov 2015 15:01:21 +1100
> 
> On 30/11/2015 8:42 AM, Kirk McKusick wrote:
>> Author: mckusick
>> Date: Sun Nov 29 21:42:26 2015
>> New Revision: 291460
>> URL: https://svnweb.freebsd.org/changeset/base/291460
>> 
>> Log:
>>   As the kernel allocates and frees vnodes, it fully initializes them
>>   on every allocation and fully releases them on every free.  These
>>   are not trivial costs: it starts by zeroing a large structure then
>>   initializes a mutex, a lock manager lock, an rw lock, four lists,
>>   and six pointers. And looking at vfs.vnodes_created, these operations
>>   are being done millions of times an hour on a busy machine.
>>   
>>   As a performance optimization, this code update uses the uma_init
>>   and uma_fini routines to do these initializations and cleanups only
>>   as the vnodes enter and leave the vnode_zone. With this change the
>>   initializations are only done kern.maxvnodes times at system startup
>>   and then only rarely again. The frees are done only if the vnode_zone
>>   shrinks which never happens in practice. For those curious about the
>>   avoided work, look at the vnode_init() and vnode_fini() functions in
>>   kern/vfs_subr.c to see the code that has been removed from the main
>>   vnode allocation/free path.
>>   
>>   Reviewed by: kib
>>   Tested by:   Peter Holm
>> 
>> Modified:
>>   head/sys/kern/vfs_subr.c
>> 
> 
> Kirk,
> 
> Very interesting.
> 
> Any estimation / expectation on the performance benefit of this,
> and in what scenario's / use-cases it might be particularly
> beneficial? Any benchmarks or tests you can share?
> 
> ./koobs

The senario in which it will be beneficial is one in which many
files are being accessed (independent of filesystem type(s) on
which they are being accessed). The key value to check is:

	sysctl vfs.vnodes_created

This is the count of vnode creations that are no longer incurring
the creation/free overhead. If this number is large and/or growing
quickly, you are getting the savings. An example that will drive
this number through the roof is:

	find / -type f -ls >/dev/null

Most systems have enough vnodes that they can cache all the files
that they access (see `sysctl kern.maxvnodes'). But when the cache
is not big enough, this change will ease the pain of reallocating
vnodes.

	Kirk McKusick