Continual benchmarking / regression testing?

[Alan Somers]

On Wed, Jan 8, 2014 at 1:28 AM, Erik Cederstrand
<erik+lists at cederstrand.dk> wrote:
> Den 07/01/2014 kl. 17.09 skrev Ivan Voras <ivoras at FreeBSD.org>:
>
>> Hello,
>>
>> Is someone working on a contitual benchmarking / regression testing
>> project for FreeBSD? I seem to recall there was a post several months
>> ago but I can't find it.
>
> I have an old project lying with a build/PXE server, a client installer and benchmark runner and a website presenting benchmark reports. It’s a bit outdated, but maybe it can be a starting point? It’s a mix of shell and Python. See https://github.com/ecederstrand/perfmon.

I like that you stored test results in a SQL database.  My SQL-foo is
poor, so ATM my framework is using CSV.  It also looks like you've got
code to generate a website.  Do you have any example output?  I never
got to that part.  The PXE stuff, however, does not belong in the
benchmark framework, IMHO.  I think that the benchmark framework
should just include the benchmarking and system profiling aspect, not
system configuration.  Provisioning and configuring systems can be
done in a separate utility, one that can be shared, for example, with
the continuous Kyua tester.

>
> I’d like to work with you on this, if you want.

I'll talk to the boss today to see if I can get any time for
benchmarking, or at least to open our proprietary code.

-Alan

>
> Erik
> _______________________________________________
> freebsd-hackers at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe at freebsd.org"