FILEDESC_LOCK() implementation

[Paul Allen]

>From Robert Watson <rwatson at freebsd.org>, Wed, Jun 21, 2006 at 07:46:33PM +0100:
> I would optimize very carefully here, the trade-offs are tricky, and we may 
> find that by making locking more complex, we cause cache problems, increase 
> lock hold periods, etc, even if we decrease contention.  I've wondered a 
> bit about a model where we loan fd's to threads to optimize repeated access 
> to the same fd by the same thread, but this mostly makes sense in the 
> context of a 1:1 model rather than an m:n model.
I apologize for not understanding all of the uses of the FILEDESC lock but,
isn't the more obvious partitioning per-cpu: each cpu may allocate from a 
range of fd, which cpu cache used depends on where the thread happens 
to be running.  When closing a fd, it is returned to the local (possibly 
different cpu cache).  A watermark is used to generate an IPI message to
rebalance the caches as needed.