Performance Tracker project update

[Erik Cederstrand]

Kris Kennaway wrote:
> Robert Watson wrote:
>>
>> Yes -- I was mostly thinking about backdating in order to play 
>> "catchup" when a new benchmark is introduced.
> 
> One thing I am looking at is how to best create a library of world 
> tarballs that can be used to populate a nfsroot (or hybrid of periodic 
> tarballs + binary diffs to save space).  Then you could provide your 
> benchmark in a standardized format (start/end/cleanup scripts, etc) and 
> tell a machine "go and run this benchmark on every daily snapshot for 
> the last year and give me the numbers".

That's basically what my server does. It creates world/kernel tarballs 
(around 90MB), dumps them in an NFS exported folder and adds them to a 
queue for the slave to consume. The slave installs the tarball on the 
local disk (via PXE) in less than 2 mins and runs whatever benchmarks it 
was told. I can pretty easily add more benchmarks later and let the 
slave collect the data using the tarballs I already created. Currently 
the benchmark script is contained within the tarball, so each tarball 
would need to be opened and closed to replace the script. I'd like to 
change that.

It gets a bit complicated when the benchmark depends on another 
application (e.g. super-smack and mysql). Currently I compile mysql from 
scratch on every new world using whichever version was the latest in the 
ports tree at the time. This has some disadvantages.

Erik