A new way to test systems in multiple machine scenarios...

[Garrett Cooper]

On Jul 6, 2014, at 9:06 PM, Craig Rodrigues <rodrigc at FreeBSD.org> wrote:

> On Sat, Jul 5, 2014 at 8:04 PM, George Neville-Neil <gnn at neville-neil.com>
> wrote:
> 
>> Hi,
>> 
>> I've coded up a system to allow you to control multiple other systems for
>> use in testing.
>> 
>> https://github.com/gvnn3/conductor
>> 
>> 
> Cool!  The architecture you have is similar to that of the SPECsfs
> benchmark test (  http://www.spec.org/sfs2008/ )
> which involves a "coordinator node" and multiple "client nodes" which
> direct NFS network
> traffic towards a System Under Test (SUT).  Garrett Cooper actually set up
> the original testbed
> that I am using now at iXsystems. :)
> 
> It would be cool to put together tools like Jenkins, Kyua, and conductor to
> do more advanced testing
> of FreeBSD before the project puts out releases.

Agreed. The only thing that I have some concern about is the reinventing of the wheel in python. multiprocessing Managers are one viable option that’s existed since python 2.6; there’s a learning curve though, and you’ll run into problems with pickling some objects because the pickle protocol is far from complete (example: http://stackoverflow.com/questions/1816958/cant-pickle-type-instancemethod-when-using-pythons-multiprocessing-pool-ma ); you might run into this problems regardless because you’re serializing objects using pickle instead of using dill (or using a simpler serialization method like JSON). Fabric has a framework that’s nice to use if you have ssh capability. There are other frameworks that use twisted conch I think too (another library that implements ssh access).

Isilon has a framework they use, but it’s very customized to their infrastructure and product assumptions and it’s in need of an overhaul :(.

-Garrett