Re: pjdfstest integration

On Tue, Apr 21, 2026 at 10:44:39AM -0600, Alan Somers wrote:
> On Tue, Apr 21, 2026 at 10:41 AM Mark Johnston <markj@freebsd.org> wrote:
> >
> > On Tue, Apr 21, 2026 at 09:41:31AM -0600, Alan Somers wrote:
> > > On Tue, Apr 21, 2026 at 9:11 AM Mark Johnston <markj@freebsd.org> wrote:
> > > >
> > > > Hi, I noticed that we install pjdfstest to /usr/tests/sys/pjdfstest,
> > > > but:
> > > > - it's not hooked up to the test suite, i.e.,
> > > >   "kyua test -k /usr/tests/Kyuafile" doesn't run it,
> > > > - contrib/pjdfstest doesn't seem to be updated regularly,
> > > > - the configuration is hard-coded, i.e., I can't easily run it against a
> > > >   filesystem of my choice.
> > > >
> > > > How hard would it be to parameterize the tests so that we can run the
> > > > tests again a list of filesystems?  For each filesystem we'd have some
> > > > little script that sets up some scratch space, creates an empty
> > > > filesystem and points pjdfstest at it.  In some cases we'd need the test
> > > > runner to specify some additional variables, e.g., for p9fs you want the
> > > > test runner to provide a share, as we currently only support the virtio
> > > > transport.  I'm not sure if kyua can pass variables to a TAP test, so
> > > > the solution might be to wrap each pjdfstest run with an ATF test case
> > > > which handles the setup.
> > > >
> > > > Is anyone interested in working on these things?
> > >
> > > Yes, yes and yes.
> > >
> > > I was indeed working on a change to pjdfstest which, among other
> > > things, would read a config file for each file system under test.  The
> > > config file specifies things like whether posix_fallocate is supported
> > > on that file system and which file flags are supported.  The change
> > > also drastically speeds up pjdfstest's runtime.
> > >
> > > We did that as part of GSoC 2022.  The status of the project is that
> > > it's 99% complete, but requires somebody to comb through 4000 SLOC
> > > line by line to make sure nothing got left out.  That's very tedious,
> > > which is why nobody has done it yet.  I would LOVE to get it finished,
> > > but I've never made the time.
> > > The rewrite also relies on some ugly macro syntax.  We did that
> > > deliberately to save time, but it does make the code ugly, and a
> > > little bit harder to review.  It might be worth investing the time to
> > > rewrite those macros more cleanly.
> > >
> > > Using the new pjdfstest, it would be quite easy to add an ATF test for
> > > each file system.  atf-sh would format the file system under test,
> > > then call pjdfstest with the appropriate per-filesystem config file.
> >
> > Do you have a pointer to this work anywhere?  I can't promise to
> > complete it, but I'm pretty motivated to stand something up for p9fs.
> 
> It's at git@github.com:musikid/pjdfstest.git .  What do you think is
> the best path forward?  I could publish a 0.1 release now, and get
> this into ports, while you work on the ATF part.  Then we could slowly
> open a series of reviews that delete the old sh-based stuff.

That sounds fine to me.  I'm not really set up to review the
implementation, but I'll try writing some integration scripts.  I have a
couple of questions though:
- Do you know of any requirements on the filesystem under test?
  Specifically, how big does it need to be?  I'm wondering if we want to
  use md(4) disks to provide the backing store for each FS, or whether
  we should rely on the test harness to provide some raw devices.
- I noticed that pjdfstest (both old and new) don't seem to execise
  getdirentries(2) at all.  Is there any particular reason for that?