DTrace script to trace processes entering vfs::vop_remove

['Mark Johnston']

On Wed, Dec 03, 2014 at 06:03:45PM -0800, dteske at FreeBSD.org wrote:
> 
> 
> > -----Original Message-----
> > From: Mark Johnston [mailto:markjdb at gmail.com] On Behalf Of Mark
> > Johnston
> > Sent: Wednesday, December 3, 2014 4:45 PM
> > To: dteske at FreeBSD.org
> > Cc: freebsd-dtrace at freebsd.org; 'Julian Elischer'
> > Subject: Re: DTrace script to trace processes entering vfs::vop_remove
> > 
> > On Wed, Dec 03, 2014 at 03:19:31PM -0800, dteske at FreeBSD.org wrote:
> > > Hi markj, list,
> > >
> > > I wrote a script for $work to help me find out "who on Earth
> > > keeps deleting files XYZ?" from a particular storage server.
> > >
> > > Please find attached a copy of watch_vop_remove.d which
> > > has the following sample output:
> > >
> > > 2014 Dec  3 11:58:52 rm[75596]: /tmp/foo
> > >  -+= 72846 0.0 -bash
> > >   \-+= 75589 0.0 /bin/bash /usr/home/support/bash_script
> > >     \-+= 75596 0.0 rm -f /tmp/foo
> > >
> > > The above sample output was displayed when executing the following shell
> > > script:
> > >
> > > #!/bin/bash
> > > touch /tmp/foo
> > > rm -f /tmp/foo
> > >
> > > The output format displayed for each vop_remove() call is as follows:
> > >
> > > DATE process[PID]: PATH_TO_DELETE
> > >  -+= GPID UID.GID grandparent_process [arguments (up to 3)]
> > >   \-+= PPID UID.GID parent_process [arguments (up to 3)]
> > >     \-+= PID UID.GID process [arguments (up to 3)]
> > 
> > This is neat. I just had a few comments:
> > - You can use walltimestamp when printing the date and time, instead of
> >   timestamp + blah.
> 
> I read that online as well, however:
> walltimestamp appears to _always_ be zero.

Right, it wasn't working properly on 8.0. :(

gnn committed a fix for that as r238537.

> 
> 
> > - It's possible to get the full argv of the current process with
> >   curpsinfo->pr_psargs. It can be done for other processes too; see
> >   /usr/lib/dtrace/psinfo.d. (This might not be true depending on the
> >   FreeBSD version you're on.)
> 
> Thanks! I'll have a look.
> 
> > - Running this script with a make -j4 buildkernel causes dtrace to run
> >   out of dynamic variable space.
> > 
> 
> Any recommendation on how to fix that?
> 
> #pragma D option dynvarsize=what_exactly?
> (16m causes a warning that it's lowering the dynamic variable memory)

It looks like a leak - once I start seeing the errors, no file removals
are logged at all. Dynamic variables need to be set to 0 once they're
finished with in order to release the consumed memory.

> 
> 
> > I'd really really like to fix name resolution so that we don't have to
> > jump through so many hoops to write scripts like this, though. One
> > approach is to do what Solaris does, which is keep a cached path in the
> > vnode itself (v_path).
> > 
> 
> Yes, that would be great. But perhaps not something we should
> do solely for dtrace's benefit.
> -- 
> Devin
>