Impact of having a large number of open file descriptors

Garance A Drosihn drosih at rpi.edu
Tue Jun 3 02:21:28 UTC 2008


At 12:33 AM +0200 6/3/08, Kris Kennaway wrote:
>Ivan Voras wrote:
>>Suleiman Souhlal wrote:
>>
>>>I have an old patch that makes kqueue monitor every file write on 
>>>the system and return the inode number in the knote's data field: 
>>>http://people.freebsd.org/~ssouhlal/testing/kqueue-anyvnode-20050503.diff 
>>>.
>>>
>>>I'd think it shouldn't be too hard to make it per-mountpoint..
>
>FWIW, I would love to use this.  I have situations where I have huge 
>numbers of files and need to cheaply detect changes so I can 
>resynchronize them to remote machines.

I remember a discussion of changes to MacOS10 in Leopard which made 
it easier to implement features such as Spotlight and TimeMachine. 
The description starts here, I think:

http://arstechnica.com/reviews/os/mac-os-x-10-5.ars/7

the section on file-system events.

The idea I thought was interesting was to save the metadata on a 
directory basis, instead of saving it on the file.  So, if file 
/some/dir/fname was changed, then they'd record that *some* file 
under /some/dir has changed.

So when your userland process comes along later on, it still has to 
scan all files in that directory to see which file(s) actually 
changed.  But that's a lot less work than scanning all files in the 
filesystem, and it also means there is much less data that has to be 
kept track of.

I have no idea how easy it would be to implement something similar on 
FreeBSD, but the strategy seemed like a pretty neat idea.

-- 
Garance Alistair Drosehn            =   gad at gilead.netel.rpi.edu
Senior Systems Programmer           or  gad at freebsd.org
Rensselaer Polytechnic Institute    or  drosih at rpi.edu


More information about the freebsd-hackers mailing list