Re: native inotify implementation

On Sat, May 17, 2025 at 8:19 AM Mark Johnston <markj@freebsd.org> wrote:
>
> On Fri, May 16, 2025 at 11:02:33AM -0500, Jake Freeland wrote:
> > On Mon May 12, 2025 at 3:58 PM CDT, Mark Johnston wrote:
> > > For the past while I've been hacking on a native implementation of
> > > Linux's inotify.  Functionality-wise, this is similar to but not quite
> > > equivalent to the EVFILT_VNODE kqueue filter.  While we already have a
> > > userspace implementation of inotify built on top of kqueue, it shares
> > > the limitations of EVFILT_VNODE, and my version can also be used in the
> > > Linuxulator.  (Please let me know if you're interested in working on
> > > that and testing it out.)
> > >
> > > The WIP implementation is here: https://reviews.freebsd.org/D50315
> > > There are some loose ends to tie up there, but I wanted to solicit
> > > feedback before I keep spending time on it.  I also wonder how this
> > > feature should be handled in the ports tree where libinotify is used
> > > today: if src starts installing /usr/include/sys/inotify.h, will ports
> > > start using the native implementation automatically?  Do we need to have
> > > some kind of flag day?
> >
> > Seems like libinotify might cause some trouble.
> >
> > See https://gcc.gnu.org/onlinedocs/cpp/Invocation.html:
> >
> > "You can use -I to override a system header file, substituting your own
> > version, since these directories are searched before the standard system
> > header file directories."
> >
> > Looks like this is true in practice for clang:
> >
> > [root@freebsd ~]$ cpp -v -I/usr/local/include /dev/null -o /dev/null
> > ...
> > clang -cc1 version 19.1.7 based upon LLVM 19.1.7 default target x86_64-unknown-freebsd15.0
> > #include "..." search starts here:
> > #include <...> search starts here:
> >  /usr/local/include
> >  /usr/lib/clang/19/include
> >  /usr/include
> > End of search list.
> >
> > Projects that use libinotify likely already specify -I/usr/local/include
> > so the C preprocessor can find /usr/local/include/sys/inotify.h. I
> > imagine they'd need to remove that include or uninstall the package to
> > pick up your new system header.
> >
> > > This work was largely motivated by a race condition in EVFILT_VNODE: in
> > > order to get events for a particular file, you first have to open it, by
> > > which point you may have missed the event(s) you care about.  For
> > > instance, if some upload service adds files to a directory, and you want
> > > to know when a new file has finished uploading, you'd have to watch the
> > > directory to get new file events, scan the directory to actually find
> > > the new file(s), open them, and then wait for NOTE_CLOSE (which might
> > > never arrive if the upload had already finished).  Aside from that, the
> > > need to hold each monitored file open is also a problem for large
> > > directory hierarchies as it's easy to exhaust file descriptor limits.
> > >
> > > My initial solution was a new kqueue filter, EVFILT_FSWATCH, which lets
> > > one watch for all file events under a mountpoint.  The consumer would
> > > allocate a ring buffer with space to store paths and event metadata,
> > > register that with the kernel, and the kernel would write entries to the
> > > buffer, using reverse lookups to find a path for each event vnode.  This
> > > prototype worked, but got somewhat hairy and I decided it would be
> > > better to simply implement an existing interface: inotify already exists
> > > and is commonly used, and has a somewhat simpler model, as it merely
> > > watches for events within a particular directory.
> >
> > I've found that more and more developers are blindly using Linux-specific
> > interfaces these days, so +1 for natively supporting another one.
> >
> > The more support we have for these, the easier porting/Linux emulation is.
> > I think the benefits of this far outweighs the cost of maintaining the
> > code.
>
> I think so too.  My perspective is that we should implement widely used
> Linux interfaces as part of the larger goal of making existing software
> usable on FreeBSD.  This is more important than the purity of the
> kernel's interfaces or architecture, at least up to a certain point.
Yes, I agree with this. I finally decided to try and run a FreeBSD laptop
and was surprised how easily it was set up. (Thanks go to the folk that
wrote that section of the Handbook. 0mp@ and others, maybe?)

rick

>
> The whole purpose of an OS is to let users run the programs they want to
> run, without getting in the way (too much).
>
> > Let me know if you need any help testing this.
>
> Thanks.  I'll follow up on in this thread once the patch is a bit closer
> to being ready.  Right now, Gleb is kindly helping run various test
> suites and comparing the native inotify implementation with libinotify.
>