Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

Thu Dec 22 18:49:56 UTC 2011

(Resending - hopefully without horrific escaping somewhere upstream).

Let me suggest an alternative.

Within the Phoronix Test Suite ecosystem, we have a continious 
integration/validation system called Phoromatic 
(http://www.phoromatic.com/).  We have a brief theory of operation on it 
captured https://docs.google.com/present/view?id=ds439pg_42hpg57m86 . 
You can see a Linux oriented tracker available at 
http://phoromatic.com/kernel-tracker.php (the nice graphs are when you 
select 180 days and the ION 330 systems).  We've re-assigned the systems 
to other projects so it is no longer being updated, but it served our 
purposes.

This system will allow you to just dedicate a machine to be updateable 
and pick up the directions for which test to be run.  The FreeBSD 
project (or contributors) would maintain the slave or test machines.  
The test suite would be selected and managed by the FreeBSD project or 
contributors.  The glue code to emit triggers (possibly SVN, git or 
other submission, or even just daily) and the scripts to update the 
systems would also be maintained by the FreeBSD community.

Phoronix Media would be happy to host it on Phoromatic.com (we've played 
with hosted with project branding) and provide the data store and the 
analytics.  We'd also be willing to make enhancements to support the 
FreeBSD project.

This should solve the "I don't have time to maintain a automated test 
infrastructure".  You don't need to, just write the glue scripts, and 
dedicate a couple of machines.  I believe FreeBSD vendors like ixSystems 
may be able to support this effort with a dedicated machine.  You can 
have as many machines as you like demonstrating AMD/Intel/32/64/large 
mem/low mem,etc.

The comments around interactivity can also be measured to some extent.  
We have the model of a "monitor".  This can be configured to determine 
jitter around a number of system variables and to possibly inject 
actions to measure impacts.

We're more than happy to work with you guys, and are willing to help do 
a lot of the infrastructure and automation lifting.

Regards,

Matthew

On 12/22/11 8:56 AM, Adrian Chadd wrote:
> Guys, girls, fuzzy creatures,
>
> This is by far the best example of a constructive email in this entire thread.
>
> If people would like to help, Erik here is exactly the kind of person
> with exactly the kind of software that needs a hand.
>
> I think enough philosophizing has been done - now we have questions
> that need answering; theories that need testing. And that requires,
> you know, coding. :)
>
> The best thing right now would be for *BSD people to pick up the
> Phronix test suite, try to compile/run it, and provide feedback. Do
> your own benchmarks on your own hardware and report back the results.
> That's how we fix the "benchmarking problem." We don't fix it by
> armchair philosophy, we fix it by getting our hands dirty. :)
>
> 2c,
>
>
> Adrian
>
>
> On 22 December 2011 03:21, Erik Cederstrand<erik at cederstrand.dk>  wrote:
>> Den 21/12/2011 kl. 19.48 skrev Alexander Leidinger:
>>
>>> And related to the subject: wasn't it you who developed the automatic benchmarking stuff? If yes, why not make it available? If you don't have he resources, I offer my help to make it available somewhere.
>> Yes, that's me. I'm mostly out of time right now, but I'd like to offer help if someone wants to pick up the project.
>>
>> For those who haven't heard about it, it's a system designed specifically to track performance of FreeBSD over time by comparing revisions of FreeBSD, everything else being equal. It consists of a tinderbox-like build script for a build server, a script to install FreeBSD and run benchmarks on at least one slave, and a database-backed website to aggregate and visualize results.
>>
>> The framework does work as-is, but it really needs to be updated: convert the scripts to use the SVN repo instead of CVS, improve visualization and search on the web fronted, and improve the benchmarking script so it's easier to extend. I don't have hardware available to run the benchmarks, but I think there's hardware available in the FreeBSD cluster.
>>
>> Here's a link to the source code:http://dev.affect-it.dk/tracker.tgz
>> And to my thesis describing how it works:http://dev.affect-it.dk/tracker.pdf
>>
>> Just send me a mail if you're interested.
>>
>> Thanks,
>> Erik_______________________________________________
>> freebsd-performance at freebsd.org  mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
>> To unsubscribe, send any mail to"freebsd-performance-unsubscribe at freebsd.org"
> _______________________________________________
> freebsd-performance at freebsd.org  mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-performance
> To unsubscribe, send any mail to"freebsd-performance-unsubscribe at freebsd.org"