kqueue of a nfs mounted file not working

[Konstantin Belousov]

On Mon, Nov 16, 2015 at 09:00:09AM -0500, Rick Macklem wrote:
> Daniel Braniss wrote:
> > 
> > > On 15 Nov 2015, at 17:26, Konstantin Belousov <kostikbel at gmail.com> wrote:
> > > 
> > > On Sun, Nov 15, 2015 at 11:22:55AM +0200, Daniel Braniss wrote:
> > >> HI,
> > >> I???m writing a program to monitor a file using kqueue(2), if the file is
> > >> local
> > >> all is OK, but if the file is via a nfs mounted fs, it only works once.
> > >> stat shows the file growing, but kevent is not triggered.
> > > 
> > > Does file grow due to local changes on the nfs client, or some other
> > > client changes the file, while your client tries to get kevent
> > > notifications ?
> > 
> > it gets updated by a host which has the file as local, so yes, it gets
> > updated
> > by another client/host.
> > 
> Hmm, I am not surprised that this doesn't work. The only indication to the
> client that the file has changed on the server is a change in the file's
> attributes when they're acquired (via a Getattr RPC or similar) from the server.
> 
> There is a vfs operation called VFS_SYSCTL(). This isn't implemented on
> the current NFS client. It was implemented on the old one, but only for
> NFS locking events and I didn't understand what needed to be done, so I
> didn't do it.
You probably mean VOP_KQFILTER, not VFS_SYSCTL().  BTW, the later is
only used by nfs and I do not quite see why its functionality not subsumed
by the mount options.

WRT VOP_KQFILTER, the default implementation is adequate.  What is missed
for NFS is the knote activation when the code notes that the cached
metadata is invalidated by server.

> Kostik, do you know if there is a VFS_SYSCTL() call done when the kevent
> stuff is probing for a file size change? (Or does it not probe and events
> get triggered via the write syscall or ???) I took a quick look at the kevent
> stuff, but admit I got lost and couldn't figure out what triggered events
> being logged?
> 
> Also, is the event for "file growing" or "file changed"?
> If it is the latter, all the NFS client can do is look for a change in
> the file's modify time and this is often at a resolution of 1sec., which
> implies that a change within the same second as the previous one may not
> be noticed. (NFSv4 has a Change attribute that is always guaranteed to
> change, but that is only NFSv4.) Also, you see metadata changes as well
> as data changes, at least for the NFSv4 attribute.

Please look at the sys/kern/vfs_subr.c lines 4296-4419.  There is a bunch
of the post-VOP hooks which are executed after the filesystem VOP method
was executed.  You would see the complete list of the notifications which
are sent, and the way they are sent, by calling VFS_KNOTE{_LOCKED}(vp,
NOTE_XXX).  Similar calls should be spread over the nfs client code
when the attribute cache entries are deleted, possibly comparing old
and new values to select proper notification.

But you are of course right that client cannot fully implement the
notifications without the server notifying it, so whatever efforts
are done for NFSv3, there are definitely will be cases which cannot
work. I do not know NFSv4 enough to make similar statement, but I think
something would prevent the complete implementation, e.g. for rename.

This makes me wonder should we need to add the calls to VFS_KNOTE() into
the nfs client, at all. It is similar to lockd/nolockd options, in
that the client might be not interested in the server or other clients
updates, only in the local notifications.  This plus the fact that the
feature cannot have complete implementation, raises the question.