Re: native inotify implementation

On Sat, 17 May 2025 11:18:34 -0400
Mark Johnston <markj@freebsd.org> wrote:

> On Fri, May 16, 2025 at 11:02:33AM -0500, Jake Freeland wrote:
> > On Mon May 12, 2025 at 3:58 PM CDT, Mark Johnston wrote:  
> > > For the past while I've been hacking on a native implementation of
> > > Linux's inotify.  Functionality-wise, this is similar to but not quite
> > > equivalent to the EVFILT_VNODE kqueue filter.  While we already have a
> > > userspace implementation of inotify built on top of kqueue, it shares
> > > the limitations of EVFILT_VNODE, and my version can also be used in the
> > > Linuxulator.  (Please let me know if you're interested in working on
> > > that and testing it out.)
[...]
> > > This work was largely motivated by a race condition in EVFILT_VNODE: in
> > > order to get events for a particular file, you first have to open it, by
> > > which point you may have missed the event(s) you care about.  For
> > > instance, if some upload service adds files to a directory, and you want
> > > to know when a new file has finished uploading, you'd have to watch the
> > > directory to get new file events, scan the directory to actually find
> > > the new file(s), open them, and then wait for NOTE_CLOSE (which might
> > > never arrive if the upload had already finished).  Aside from that, the
> > > need to hold each monitored file open is also a problem for large
> > > directory hierarchies as it's easy to exhaust file descriptor limits.
> > >
> > > My initial solution was a new kqueue filter, EVFILT_FSWATCH, which lets
> > > one watch for all file events under a mountpoint.  The consumer would
> > > allocate a ring buffer with space to store paths and event metadata,
> > > register that with the kernel, and the kernel would write entries to the
> > > buffer, using reverse lookups to find a path for each event vnode.  This
> > > prototype worked, but got somewhat hairy and I decided it would be
> > > better to simply implement an existing interface: inotify already exists
> > > and is commonly used, and has a somewhat simpler model, as it merely
> > > watches for events within a particular directory.  
> > 
> > I've found that more and more developers are blindly using Linux-specific
> > interfaces these days, so +1 for natively supporting another one.
> > 
> > The more support we have for these, the easier porting/Linux emulation is.
> > I think the benefits of this far outweighs the cost of maintaining the
> > code.  
> 
> I think so too.  My perspective is that we should implement widely used
> Linux interfaces as part of the larger goal of making existing software
> usable on FreeBSD.  This is more important than the purity of the
> kernel's interfaces or architecture, at least up to a certain point.
> 
> The whole purpose of an OS is to let users run the programs they want to
> run, without getting in the way (too much).

Yes, and no. While it's often useful in short-term perspective, such approach
leaves FreeBSD without unique features so it becomes yet another "Linux, just
poorer" with obvious then "why choose it?". It's understandable that in some
cases it is simple to implement compatible API, but an alternative like "have
more general solution with a compatibility shim layer via which their API is
implemented" is better, when possible.

It's late in which particular topic as commit was landed, but for future we
should think how to extend kqueue to be able more.
[E.g. I'd want to have notifications for my protocol with multiple streams
inside one socket (think like QUIC), but it does not fit nicely into current
struct kevent or socket API (multiple socket buffers with separate reading)]


-- 
WBR, @nuclight