[BUG] Getting path to program binary sometimes fails

[John Baldwin]

On Friday, December 05, 2014 03:52:41 PM Mike Gelfand wrote:
> On Dec 5, 2014, at 6:19 PM, John Baldwin <jhb at freebsd.org> wrote:
> >> No, not NFS but ZFS. Could that be an issue? The FreeBSD 8 machine I
> >> mentioned before has UFS.
> >> 
> >> Also, as you can see from the video I recorded (and from the code I
> >> provided), path resolution succeeds and fails within fractions of a
> >> second
> >> after process startup.
> > 
> > Are you seeing vnodes being actively recycled?  In particular, do you see
> > vfs.numvnodes close to kern.maxvnodes?  You can try raising
> > kern.maxvnodes.
> > If vfs.numvnodes grows up to the limit then as long as you can stomach the
> > RAM of having more vnodes around that would increase the changes of your
> > paths remaining valid.
> 
> When the call works, sysctl returns:
>     vfs.numvnodes: 59638
>     kern.maxvnodes: 204723
> The times it doesn't, the output is:
>     vfs.numvnodes: 60017
>     kern.maxvnodes: 204723
> I've selected maximum numbers. Monitoring was made with
>     while sysctl vfs.numvnodes kern.maxvnodes; do sleep 0.1; done
> 
> So it seems that's not related, correct? 60K is much less than 200K.

Yes.  Unfortunately, we don't expose a raw counter for vnode recycling (I
should really add one).  I think we might also purge unused vnodes if we have
too many "free" vnodes.  However, directories that aren't currently open by
a process (meaning via opendir()) count as "unused", so are subject to
purging.  You can try increasing "vfs.wantfreevnodes".

Also, please try this patch.  It just adds a counter for recycled vnodes.  If 
this value increases during your test then it does show that recycling is 
occurring.  If it doesn't, then that rules it out.

Index: vfs_subr.c
===================================================================
--- vfs_subr.c	(revision 275512)
+++ vfs_subr.c	(working copy)
@@ -156,6 +156,10 @@ static int vlru_allow_cache_src;
 SYSCTL_INT(_vfs, OID_AUTO, vlru_allow_cache_src, CTLFLAG_RW,
     &vlru_allow_cache_src, 0, "Allow vlru to reclaim source vnode");
 
+static u_long recycles_count;
+SYSCTL_ULONG(_vfs, OID_AUTO, recycles, CTLFLAG_RW, &recycles_count, 0,
+    "Number of vnodes recycled");
+
 /*
  * Various variables used for debugging the new implementation of
  * reassignbuf().
@@ -988,6 +992,7 @@ vtryrecycle(struct vnode *vp)
 		    __func__, vp);
 		return (EBUSY);
 	}
+	atomic_add_long(&recycles_count, 1);
 	if ((vp->v_iflag & VI_DOOMED) == 0)
 		vgonel(vp);
 	VOP_UNLOCK(vp, LK_INTERLOCK);

-- 
John Baldwin