Monitoring FS changes

Mon Jan 4 15:03:41 UTC 2016

On 4/01/2016 5:02 PM, Konstantin Belousov wrote:
> On Sun, Jan 03, 2016 at 01:36:36PM -0800, Jordan Hubbard wrote:
>>> On Jan 3, 2016, at 1:08 PM, Mark Felder <feld at FreeBSD.org> wrote:
>>>
>>> If someone, anyone out there is capable of bringing us something that
>>> does scale it would be greatly appreciated. Lots of nice Linux software
>>> uses this, but when they do port to FreeBSD we have to do full
>>> filesystem scans. It's such a waste.
>> I???ve been pondering this for awhile since a lot of interesting enterprise features require a working filesystem change notification mechanism that scales to thousands or even millions of files (how did we bump into the 32 bit NFS file handle problem at iXsystems?  Somebody tried to share more than 4 billion files over NFS - Enterprise folks do some crazy s**t!).
>>
>> The big question is less whether it???s possible and more what kind of mechanism people will find palatable.  The OS X FSEvents mechanism works reasonably well and is used constantly to trigger things like spotlight search indexing and such, and I was by no means involved in its creation at Apple so I can only speak peripherally to the implementation, but it seems like it took a fairly long time for it to become ???light weight??? enough to use without the overhead being punitive.  Any similar mechanism in FreeBSD would also have to go through some evolutionary performance iterations - do people want it badly enough to invest in it long-term?  I don???t know, but I do know that a long-term investment would be necessary to really make it work well and provide all of the appropriate APIs for talking to it.
>>
>> I think we can probably all agree that Linux inotify wouldn???t be worth the trouble.  From the wikipedia page:
>>
>> 	??? Inotify does not support recursively watching directories, meaning that a separate inotify watch must be created for every subdirectory.[4]
>> 	??? Inotify does report some but not all events in sysfs and procfs.
>> 	??? Notification via inotify requires the kernel to be aware of all relevant filesystem events, which is not always possible for networked filesystems such as NFS where changes made by one client are not immediately broadcast to other clients.
>> 	??? Rename events are not handled directly; i.e., inotify issues two separate events that must be examined and matched in a context of potential race conditions.
>>
>> I think the first issue alone is a deal killer.  Having to walk the filesystem tree posting notifications on every [new] directory just to watch a filesystem in its entirety would be pretty onerous and failure-prone to boot.  By contrast:  https://en.wikipedia.org/wiki/FSEvents
>>
>> This is also not to say that I would expect anything in FreeBSD to be API-compatible (though the upstream clients would probably grumble at yet another notification mechanism API to #ifdef into their code), simply that there are only so many design patterns to follow.  A filesystem change is a filesystem change.  Everything beyond that is just a glorified pub/sub mechanism.
>>
>> Assuming there???s interest, I could potentially see throwing some engineering effort into this.
>>
> There are many people that claim to have very good ideas. This case
> seems to be an opportunity for such people to contribute something
> useful to the FreeBSD.
>
> I mean, agree upon and provide the precise enough technical
> specification for the API you want. It does not need to be exact in all
> details, you can assume a gleam of intelligence in the coders which
> would implement it, but the spec must be feasible to implement and
> satisfy the core requirements for consumers.
>
> E.g., you need a recursive notification, but there is no way to find
> all dirents pointing to the given inode, as you noted above. NFS client
> should not expect to (reliably) get a notification when other client
> updates a directory and so on.
>
> Ideally, this should be a man-page like text and several programs to
> illustrate the intended use.  Programs should be complete, but cannot
> be tested (for understandable reason).
>
> After  that, I promise that the spec will be implemented.
I think the point is that setting  a notify onto a directory vnode 
would force all directory vnodes above it (away from root)
to stay in memory. That gives a path to traverse in memory to look for 
notify hooks when a change is made..
This has been discussed many times before.  many yearsa ago it came 
down to resources usage.. I'm not sure that is so important with 128GB 
machines.
(but it needs to handle runaway resource usage). The exact syntax of 
what is required needs to be spelled out well (as kib says).


> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>