ULE scheduling oddity

[Stephen Montgomery-Smith]

Steve Kargl wrote:
> It appears that the ULE scheduler is not providing a fair 
> slice to running processes.
> 
> I have a dual-cpu, quad-core opteron based system with
> node21:kargl[229] uname -a
> FreeBSD node21.cimu.org 8.0-CURRENT FreeBSD 8.0-CURRENT #3:
> Wed Jun  4 16:22:49 PDT 2008   kargl at node10.cimu.org:src/sys/HPC  amd64
> 
> If I start exactly 8 processes, each gets 100% WCPU according to
> top.  If I add to additional processes, then I observe
> 
> last pid:  3874;  load averages:  9.99,  9.76,  9.43    up 0+19:54:44  10:51:18
> 41 processes:  11 running, 30 sleeping
> CPU:  100% user,  0.0% nice,  0.0% system,  0.0% interrupt,  0.0% idle
> Mem: 5706M Active, 8816K Inact, 169M Wired, 84K Cache, 108M Buf, 25G Free
> Swap: 4096M Total, 4096M Free
> 
>   PID USERNAME    THR PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
>  3836 kargl         1 118    0   577M   572M CPU7   7   6:37 100.00% kzk90
>  3839 kargl         1 118    0   577M   572M CPU2   2   6:36 100.00% kzk90
>  3849 kargl         1 118    0   577M   572M CPU3   3   6:33 100.00% kzk90
>  3852 kargl         1 118    0   577M   572M CPU0   0   6:25 100.00% kzk90
>  3864 kargl         1 118    0   577M   572M RUN    1   6:24 100.00% kzk90
>  3858 kargl         1 112    0   577M   572M RUN    5   4:10 78.47% kzk90
>  3855 kargl         1 110    0   577M   572M CPU5   5   4:29 67.97% kzk90
>  3842 kargl         1 110    0   577M   572M CPU4   4   4:24 66.70% kzk90
>  3846 kargl         1 107    0   577M   572M RUN    6   3:22 53.96% kzk90
>  3861 kargl         1 107    0   577M   572M CPU6   6   3:15 53.37% kzk90

My personal experience is that WCPU is not that accurate a measure of 
what is really going on.  It is some kind of weighted CPU time, and 
according to the man page you have to wait for up to a minute to get an 
accurate sense.

What I tend to do is to look at the TIME's, and see how fast they tick.

Also, you can run the programs thus:

time ./kargl

and the times produced at the end tend to be a rather good measure of 
actual percentage cpu time.  Although I can see that in your situation 
that this might be tricky to use.

There is also a -C option with top that gives "raw CPU" time.  I have 
never tried it, so I cannot speak to how good it really is.

Stephen