rusage breakdown and cpu limits.

Bruce Evans brde at optusnet.com.au
Wed May 30 11:18:00 UTC 2007


On Tue, 29 May 2007, Kip Macy wrote:

>> I think using a per-process spin lock (or a pool of spin locks) would be a
>> good first step.  I wouldn't do anything more complicated unless the simple
>> approach doesn't work.  The only reason to not move the check into 
>> userret()
>> would be if one is worried about threads chewing up CPU time while they are
>> in the kernel w/o bouncing out to userland.  Also, it matters which one
>> happens more often (userret() vs mi_switch()).  If on average threads 
>> perform
>> multiple system calls during a single time slice (no idea if this is true 
>> or
>> not), then moving the check to userret() would actually hurt performance.
>
> Processes can certainly make numerous system calls within a single
> time slice.

Not many more than a few hundred million syscalls can be made within a
timeslice of 100 mS.  FreeBSD does too many context switches for
interrupts, so the number in practice seem to be mostly in the range of
1-10, but I hope for 100-1000.

> However, in userret it would be protected by a per process
> or per thread blocking mutex as opposed to a global spin mutex. It
> would be surprising if it isn't a net win, although it is quite
> possible that on a 2-way system the extra locking could have an
> adverse effect on some workloads.

Any locking within userret() would be a good pessimization.  There
are none now, but still a lot of bloat.

In this case, correct proc locking isn't even possible, since the
runtime update must occur while something like a global scheduler lock
is held.  When a context switch occurs, the lock must protect at least
the old process and the new process, and somehow prevent interference
from other processes.  The update of the runtime needs essentially the
same lock.  Any locking in userret() would need to use the same lock
as the update to be perfectly correct.  Fortunately, the cpulimit limit
check only needs to be correct to within seconds or even minutes.  A
sloppy unlocked check don't often enough would work OK, at least if
you re-check with correct locking before killing the process.
Alternatively, the sloppiness can be due to delayed updates -- let the
rusage data lag by up to a second or so in the context of the check;
the runtime would accumulate accurately somewhere, but the check
wouldn't see it all the accumulation step would need the full lock
for reading and writing the scattered data and a lesser lock for
updating the accumulated data.  userret() still shouldn't be pessimized
by acquiring the lesser lock.

I still think this misses the point -- the check is the easy part,
and can be done at no extra locking cost while the full lock is held.

Bruce


More information about the freebsd-arch mailing list