Impact of having a large number of open file descriptors
Garance A Drosihn
drosih at rpi.edu
Tue Jun 3 02:21:28 UTC 2008
At 12:33 AM +0200 6/3/08, Kris Kennaway wrote:
>Ivan Voras wrote:
>>Suleiman Souhlal wrote:
>>>I have an old patch that makes kqueue monitor every file write on
>>>the system and return the inode number in the knote's data field:
>>>I'd think it shouldn't be too hard to make it per-mountpoint..
>FWIW, I would love to use this. I have situations where I have huge
>numbers of files and need to cheaply detect changes so I can
>resynchronize them to remote machines.
I remember a discussion of changes to MacOS10 in Leopard which made
it easier to implement features such as Spotlight and TimeMachine.
The description starts here, I think:
the section on file-system events.
The idea I thought was interesting was to save the metadata on a
directory basis, instead of saving it on the file. So, if file
/some/dir/fname was changed, then they'd record that *some* file
under /some/dir has changed.
So when your userland process comes along later on, it still has to
scan all files in that directory to see which file(s) actually
changed. But that's a lot less work than scanning all files in the
filesystem, and it also means there is much less data that has to be
kept track of.
I have no idea how easy it would be to implement something similar on
FreeBSD, but the strategy seemed like a pretty neat idea.
Garance Alistair Drosehn = gad at gilead.netel.rpi.edu
Senior Systems Programmer or gad at freebsd.org
Rensselaer Polytechnic Institute or drosih at rpi.edu
More information about the freebsd-hackers