Re: pjdfstest integration

On Wed, Apr 22, 2026 at 04:27:08PM -0600, Alan Somers wrote:
> On Wed, Apr 22, 2026 at 3:43 PM Mark Johnston <markj@freebsd.org> wrote:
> >
> > On Tue, Apr 21, 2026 at 10:44:39AM -0600, Alan Somers wrote:
> > > On Tue, Apr 21, 2026 at 10:41 AM Mark Johnston <markj@freebsd.org> wrote:
> > > >
> > > > On Tue, Apr 21, 2026 at 09:41:31AM -0600, Alan Somers wrote:
> > > > > On Tue, Apr 21, 2026 at 9:11 AM Mark Johnston <markj@freebsd.org> wrote:
> > > > > >
> > > > > > Hi, I noticed that we install pjdfstest to /usr/tests/sys/pjdfstest,
> > > > > > but:
> > > > > > - it's not hooked up to the test suite, i.e.,
> > > > > >   "kyua test -k /usr/tests/Kyuafile" doesn't run it,
> > > > > > - contrib/pjdfstest doesn't seem to be updated regularly,
> > > > > > - the configuration is hard-coded, i.e., I can't easily run it against a
> > > > > >   filesystem of my choice.
> > > > > >
> > > > > > How hard would it be to parameterize the tests so that we can run the
> > > > > > tests again a list of filesystems?  For each filesystem we'd have some
> > > > > > little script that sets up some scratch space, creates an empty
> > > > > > filesystem and points pjdfstest at it.  In some cases we'd need the test
> > > > > > runner to specify some additional variables, e.g., for p9fs you want the
> > > > > > test runner to provide a share, as we currently only support the virtio
> > > > > > transport.  I'm not sure if kyua can pass variables to a TAP test, so
> > > > > > the solution might be to wrap each pjdfstest run with an ATF test case
> > > > > > which handles the setup.
> > > > > >
> > > > > > Is anyone interested in working on these things?
> > > > >
> > > > > Yes, yes and yes.
> > > > >
> > > > > I was indeed working on a change to pjdfstest which, among other
> > > > > things, would read a config file for each file system under test.  The
> > > > > config file specifies things like whether posix_fallocate is supported
> > > > > on that file system and which file flags are supported.  The change
> > > > > also drastically speeds up pjdfstest's runtime.
> > > > >
> > > > > We did that as part of GSoC 2022.  The status of the project is that
> > > > > it's 99% complete, but requires somebody to comb through 4000 SLOC
> > > > > line by line to make sure nothing got left out.  That's very tedious,
> > > > > which is why nobody has done it yet.  I would LOVE to get it finished,
> > > > > but I've never made the time.
> > > > > The rewrite also relies on some ugly macro syntax.  We did that
> > > > > deliberately to save time, but it does make the code ugly, and a
> > > > > little bit harder to review.  It might be worth investing the time to
> > > > > rewrite those macros more cleanly.
> > > > >
> > > > > Using the new pjdfstest, it would be quite easy to add an ATF test for
> > > > > each file system.  atf-sh would format the file system under test,
> > > > > then call pjdfstest with the appropriate per-filesystem config file.
> > > >
> > > > Do you have a pointer to this work anywhere?  I can't promise to
> > > > complete it, but I'm pretty motivated to stand something up for p9fs.
> > >
> > > It's at git@github.com:musikid/pjdfstest.git .  What do you think is
> > > the best path forward?  I could publish a 0.1 release now, and get
> > > this into ports, while you work on the ATF part.  Then we could slowly
> > > open a series of reviews that delete the old sh-based stuff.
> >
> > That sounds fine to me.  I'm not really set up to review the
> > implementation, but I'll try writing some integration scripts.  I have a
> > couple of questions though:
> > - Do you know of any requirements on the filesystem under test?
> >   Specifically, how big does it need to be?  I'm wondering if we want to
> >   use md(4) disks to provide the backing store for each FS, or whether
> >   we should rely on the test harness to provide some raw devices.
> 
> Puny.  I just ran it on a 16MB UFS partition and it worked fine.

Cool.

Here's some simple ATF integration: https://reviews.freebsd.org/D56605
All of the tests except the UFS1 one pass for me, some comments are in
the review.  I think the main obstacle is the need for a new test user:
why does pjdfstest need two unprivileged users?  It's already using the
"tests" user.

> > - I noticed that pjdfstest (both old and new) don't seem to execise
> >   getdirentries(2) at all.  Is there any particular reason for that?
> 
> I doubt it.  Probably PJD just never got around to writing any.
> Although, pjdfstest probably isn't the best way to test
> getdirentries(), as there are lots of corner cases that you can't
> reach without knowing a file system's internals.  But it would still
> be nice to have.

Yeah, I started looking into this since I have some local modifications
to p9fs' VOP_READDIR implementation, and I wanted to make sure they get
exercised by the test suite somehow.

> BTW, here's a related test I wrote but never committed.  It sets up a
> tmpfs file system using the code in
> ./contrib/netbsd-tests/fs/tmpfs/h_funcs.subr and runs fsx on it.  Like
> pjdfstest, fsx is file system agnostic.

Ooh, that'd be nice.  I wonder if my patch should be generalized to
allow running other test utilities?