Performance Tracker project update

[Kris Kennaway]

Erik Cederstrand wrote:
> Kris Kennaway wrote:
>>
>> This is coming along very nicely indeed!
>>
>> One suggestion I have is that as more metrics are added it becomes 
>> important for an "at a glance" overview of changes so we can monitor 
>> for performance improvements and regressions among many workloads.
>  >
>> One way to do this would be a matrix of each metric with its change 
>> compared to recent samples.  e.g. you could do a student's T 
>> comparison of today's numbers with those from yesterday, or from a 
>> week ago, and colour-code those that show a significant deviation from 
>> "no change". This might be a bit noisy on short timescales, so you 
>> could aggregrate data into larger bins and compare e.g. moving 1-week 
>> aggregates. Fluctuations on short timescales won't stand out, but if 
>> there is a real change then it will show up less than a week later.
> 
> I agree that there's a need for an overview and some sort of 
> notification. I've been collecting historical data to get a baseline for 
> the statistics and I'll try to see what I can do over the next weeks.
> 
>> These significant events could also be graphed themselves and/or a 
>> history log maintained (or automatically annotated on the individual 
>> graphs) so historical changes can also be pinpointed.
>>
>> At some point the ability to annotate the data will become important 
>> (e.g. "We understand the cause of this, it was r1.123 of foo.c, which 
>> was corrected in r1.124.  The developer responsible has been shot.")
> 
> There's a field in the database for this sort of thing. I just think it 
> needs some sort of authentication. That'll have to wait a bit.

Sounds good.

>> P.S. If I understand correctly, the float test shows a regression?  
>> The metric is calculations/second, so higher = better?
> 
> The documentation on Unixbench is scarce, but I would think so.

Interesting.  Some candidate changes from 2007-10-02:

   Modified files:
     contrib/gcc          opts.c
   Log:
   Do not imply -ftree-vrp with -O2 and above.  One must implicitly specify
   '-ftree-vrp' if one wants it.
   Some bad code generation has been tracked to -ftree-vrp.  jdk1{5,6} are
   notable examples.

     sys/kern             sched_ule.c
   Log:
    - Move the rebalancer back into hardclock to prevent potential softclock
      starvation caused by unbalanced interrupt loads.
    - Change the rebalancer to work on stathz ticks but retain 
randomization.
    - Simplify locking in tdq_idled() to use the tdq_lock_pair() rather than
      complex sequences of locks to avoid deadlock.


     sys/kern             sched_ule.c
   Log:
    - Reassign the thread queue lock to newtd prior to switching.  Assigning
      after the switch leads to a race where the outgoing thread still owns
      the local queue lock while another cpu may switch it in.  This race
      is only possible on machines where cpu_switch can take significantly
      longer on different cpus which in practice means HTT machines with
      unfair thread scheduling algorithms.

Is anyone else able to look into this?

> BTW if anyone's interested my SVN repo is online at:
> 
> svn://littlebit.dk/website/trunk    (Pylons project)
> svn://littlebit.dk/tracker/trunk    (sh/Python scripts for runnning the 
> server and slaves)
> 
> Be careful with your eyes - this is my first attempt at both shell 
> scripting and Python :-)

:)

Kris