File trees: the deeper, the weirder

[Yar Tikhiy]

On Mon, Oct 30, 2006 at 04:05:19PM +0300, Yar Tikhiy wrote:
> On Sun, Oct 29, 2006 at 11:32:58AM -0500, Matt Emmerton wrote:
> > [ Restoring some OP context.]
> > 
> > > On Sun, Oct 29, 2006 at 05:07:16PM +0300, Yar Tikhiy wrote:
> > >
> > > > As for the said program, it keeps its 1 Hz pace, mostly waiting on
> > > > "vlruwk".  It's killable, after a delay.  The system doesn't show ...
> > > >
> > > > Weird, eh?  Any ideas what's going on?
> > >
> > > I would guess that you need a new vnode to create the new file, but no
> > > vnodes are obvious candidates for freeing because they all have a child
> > > directory in use. Is there some sort of vnode clearing that goes on every
> > > second if we are short of vnodes?
> > 
> > See sys/vfs_subr.c, subroutine getnewvnode().  We call msleep() if we're
> > waiting on vnodes to be created (or recycled).  And just look at the 'hz'
> > parameter passed to msleep()!
> > 
> > The calling process's mkdir() will end up waiting in getnewvnode() (in
> > "vlruwk" state) while the vnlru kernel thread does it's thing (which is to
> > recycle vnodes.)
> > 
> > Either the vnlru kernel thread has to work faster, or the caller has to
> > sleep less, in order to avoid this lock-step behaviour.
> 
> I'm afraid that, though your analysis is right, you arrive at wrong
> conclusions.  The process waits for the whole second in getnewvnode()
> because the vnlru thread cannot free as much vnodes as it wants to.
> vnlru_proc() will wake up sleepers on vnlruproc_sig (i.e.,
> getnewvnode()) only if (numvnodes <= desiredvnodes * 9 / 10).
> Whether this condition is attainable depends on vlrureclaim() (called
> from the vnlru thread) freeing vnodes at a sufficient rate.  Perhaps
> vlrureclaim() just can't keep the pace at this conditions.
> debug.vnlru_nowhere increasing is an indication of that.  Consequently,
> each getnewvnode() call sleeps 1 second, then grabs a vnode beyond
> desiredvnodes.  It's no surprise that the 1 second delays start to
> appear after approx. kern.maxvnodes directories were created.

Sorry, I hit the send button too early, before telling what my own
conclusions were.  Well, we may try to sleep less in getnewvnode(),
or arrange the vnlru thread to wake up getnewvnode() after a failed
attempt to free more vnodes, but this will mean unlimited growth
of the number of allocated vnodes.  Perhaps we should see what
prevents vlrureclaim() from freeing enough vnodes to sustain the
vnode pressure of the pattern at issue.

-- 
Yar