Measuring ZFS configuration differences

[Dan Langille]

> On Nov 17, 2015, at 9:31 PM, Marcelo Araujo <araujobsdport at gmail.com> wrote:
> 
> 2015-11-18 10:23 GMT+08:00 Dan Langille <dan at langille.org>:
> 
>> 
>>> On Nov 17, 2015, at 8:46 PM, Marcelo Araujo <araujobsdport at gmail.com>
>> wrote:
>>> 
>>> 2015-11-18 3:14 GMT+08:00 Dan Langille <dan at langille.org>:
>>> 
>>>> 
>>>> On Nov 12, 2015, at 1:30 AM, Marcelo Araujo <araujobsdport at gmail.com>
>>>> wrote:
>>>> 
>>>> 
>>>> 
>>>> 2015-11-12 6:34 GMT+08:00 Dan Langille <dan at langille.org>:
>>>> 
>>>>> On Oct 12, 2015, at 1:00 PM, Dan Langille <dan at langille.org> wrote:
>>>>>> 
>>>>>> Following up on the discussions during EuroBSDCon 2015 (Stockholm)
>>>>> during the FreeBSD Developer
>>>>>> Summit regarding various ZFS configuration settings, I write to start
>>>>> our implementation phase now that some
>>>>>> usual suspects have joined the list.
>>>>>> 
>>>>>> re https://wiki.freebsd.org/201510DevSummit/Performance
>>>>>> 
>>>>>> I think the first order of business is granting access rights to the
>>>>> server (varm) in question:
>>>>>> 
>>>>>> http://dan.langille.org/2015/07/19/varm/
>>>>>> 
>>>>>> During the workshop, mention was made of serial access.  I can arrange
>>>>> that.
>>>>>> 
>>>>>> The server has IPMI, however, my first thought:
>>>>>> 
>>>>>> 1 - connect a USB-serial cable to varm & link that to another server
>> in
>>>>> my rack.
>>>>> 
>>>>> Marcelo: At EuroBSDCon, was it you who mentioned a particular
>>>>> configuration for the test machine which made
>>>>> it easy to configure and run tests?  Was it PXE booting or something?
>>>>> 
>>>>>> 2 - create a jail in that server and give it access to that serial
>>>>> connection
>>>>>> 3 - redirect incoming port XYZ to that jail via a public-key-only ssh
>>>>> connection
>>>>>> 4 - give people access
>>>>>> 
>>>>>> Any suggestions?
>>>>> 
>>>>> —
>>>>> Dan Langille
>>>>> http://langille.org/
>>>>> 
>>>>> 
>>>> Hello Dan,
>>>> 
>>>> Yes, was me :)
>>>> 
>>>> I mention about zopkio test framework.
>>>> I gave a presentation last weekend at PyCon Hong Kong about it.
>>>> 
>>>> Here is my slides:
>>>> 
>> http://www.slideshare.net/araujobsd/functional-and-scale-performance-tests-using-zopkio
>>>> 
>>>> The good of Zopkio is, we can write tests at once and run it as much as
>> we
>>>> want in different machines. Also Zopkio depends of Naarad, that can
>> parse a
>>>> CSV file and create metrics and SLA over those metrics, plot graphs and
>> so
>>>> on. Pretty nice tool!!!
>>>> 
>>>> I'm wondering if we could start to test something and maybe show it at
>>>> AsiaBSDCon and BSDCon(Canada) next year? What do you think?
>>>> What I need right now would be a list of tests that we want to perform
>> as
>>>> well as what parameters we would like to take as metrics to compare.\
>>>> 
>>>> 
>>>> For tests, we can start with this list:
>>>> https://github.com/dlangille/zfs_benchmarks/issues
>>>> 
>>>> We can start as soon as I figure out how to provide access to the
>>>> testers.  See above re serial connection.
>>>> 
>>>> I want to provide access, but I want to keep access restricted to only
>>>> this box and not to the rest of my home LAN.  I plan to do this via a
>>>> VLAN.
>>>> 
>>>> I could fire up a Rasperberry Pi and allow ssh into that.  Will that be
>>>> enough
>>>> power for what you need to do?
>>>> 
>>>> 
>>> First of all, thanks to share the tests cases.
>>> 
>>> If I use zopkio, the best would be access SSH direct to the target
>> machine
>>> where I need to run the tests. For zopkio, I need to have my SSH KEY on
>> the
>>> target machine.
>> 
>> I am OK with this.
>> 
>>> As I don't know your network, maybe what you could do is: Via
>> RasperBerry,
>>> forward the SSH to the target machine, I will pass-through via your
>>> RasperBerry where you can control the access for the rest of your LAN.
>>> 
>>> Another approach could be, two different subnets and a firewall. Or as
>> you
>>> said, VLANS.
>> 
>> I will be doing VLANS, which have yet to be set up.
>> 
>> The target system will have ZFS pools can be configured for different
>> tests (i.e. raidz2 vs raidz3).
>> This will involve gpart etc because the drives & pools will need to be
>> 'wiped' between different test
>> runs.
>> 
>> I seem to recall someone suggesting PXE boot and configuring the system
>> remotely.  Does anyone
>> recall that?  That aspect of the discussion was not recorded:
>> https://wiki.freebsd.org/201510DevSummit/Performance
>> 
>> 
> Bapt@ mentioned that, this is the way how we are doing in another project.
> But in my point of view, it is not a must for our case!
> 
> The PXE wold be good if we try to test different of OS flavors, or build
> different images.

OK.  The only thing holding us back is:

- adding the air filters to the case
- moving to the new switch with the new VLANs

It's now a matter of time.

— 
Dan Langille
http://langille.org/