jroberson at chesapeake.net
Thu Aug 16 06:58:27 UTC 2007
I have been looking at file locking for 8.0 and have come up with a way to
completely eliminate the file lock, reduce the size of struct file by
~4 pointers (22%), remove the global list of files and associated lock,
and restrict the scope of unp_gc while removing several race conditions.
The whole thing hinges on reducing the complexity and scope of unp_gc to
remove several fields from struct file. The remaining parts can be
protected by atomics or are already protected by other locks. f_count and
f_type are now completely updated using atomics. The ref counting with
atomics results in significantly fewer atomics and cheaper fhold/fdrop.
Protecting f_type was only complicated in cases where there were compound
operations done on it, which are now implemented with atomic_cmpset_int
The unix domain socket garbage collection was changed to scan the list of
unp sockets rather than the list of all files. This code is only
responsible for finding dead cycles of unp sockets which reference each
other. Evaluating other descriptors is not necessary. This allowed me to
move f_gcflag and f_msgcount into unpcb. This also removed a use of the
global filelist allowing me to remove two pointers from struct file.
The only negative part of the new algorithm is that a back-pointer to
the referencing struct file must be stored in any unix domain socket that
is referenced via another in a rights message. It is a slight layering
violation but there is only ever 1 file for each unix domain socket so it
is correct. This is required because the garbage collection algorithm
needs to know about the external references via the file.
The patch is available at:
pho and kris have both tested this and found it to be stable. Kris has
done some performance measurement and found it to be a win on
microbenchmarks. I can't imagine a case that would actually be slower,
except perhaps endless loops of sysctl kern.file. This also removes
several sources of contention for multithreaded applications in
particular, but also a global lock on allocating/freeing files.
This also resolves several cases where f_flag was not protected when it
should be, as well as removing race conditions in the garbage collector
code due to dropping locks, and fixing unprotected variables in the
garbage collection code.
I intend to commit this soon after the 7.0 branch is made. I also have
the final revision of my improved select locking ready.
More information about the freebsd-arch