[PATCH] Make fdescfs MPSAFE

Kris Kennaway kris at FreeBSD.org
Thu Oct 18 01:47:05 PDT 2007


Ulf Lilleengen wrote:
> On tor, okt 18, 2007 at 09:33:40 +0200, Kris Kennaway wrote:
>> Ulf Lilleengen wrote:
>>> On tor, aug 16, 2007 at 12:05:26 +0200, Ulf Lilleengen wrote:
>>>> Hi,
>>>>
>>>> To be able to better understand VFS and locking in general, I started 
>>>> making
>>>> fdescfs MPSAFE. I'm not experienced with any of these things, so there 
>>>> might be
>>>> some errors, although I've looked through much VFS code and code for 
>>>> other FS
>>>> like nullfs. I've tested it by running two pthreads on the same fd, and 
>>>> that seamt
>>>> to work, but there might be other cases where it will fail.
>>>>
>>>> Patch is attached.
>>>>
>>> Attached is a new patch that uses rwlocks instead of mutexes (since reading
>>> the hash_table is frequently done). Also, it adds checking so that there 
>>> is no
>>> duplicates in the hash table before inserting the new fdescnode. And add a
>>> mising hashfree().
>>>
>>> I also checked to see if there was any issues regarding Jeffs new patch to
>>> use atomic operations on the file structure, but there wasn't any obvious
>>> places where this affects fdescfs.
>>>
>>> Patch here:
>>> http://folk.ntnu.no/lulf/patches/freebsd/fdescfs/fdescfs_lock.diff
>> This might be OK but you should be aware that rwlocks can be slower than 
>> mutexes when there is a suitably mixed read/write workload.  We don't do 
>> the same adaptive spinning for wlocks as for mutexes when they are held 
>> by shared holders (since we don't track who they are so can't track 
>> whether they're running), and it is possible for readers to starve writers.
>>
>> If possible some benchmarks trying to find the worst case behaviour 
>> would be useful.
> Good point. I guess having a benchmark where one would open #hashentries
> files, and then have #hashentries threads reading (accessing the files) and
> one thread writing (closing perhaps) could produce the starvation?

Perhaps, give it a try :)

> I got the impression from locking(9) that a mutex was essentially a wrwlock,
> but if that's not exactly true...

Conceptually yes, but as mentioned wlocks do not have some of the 
optimizations that are important for mutexes.

 > then I don't think this part is very heavy
> contested anyway, so perhaps changing it back to a mutex saves me for a lot
> of trouble. (nullfs does this with mutexes btw).

Well, I wouldn't suggest giving up without trying.  It is always 
important to benchmark SMP changes no matter what you are doing (e.g. 
try to find the worst case scenario and make sure it's not worse), 
because sometimes there are effects you didnt think of.

Kris


More information about the freebsd-fs mailing list