Measuring ZFS configuration differences

Fri Nov 20 16:22:24 UTC 2015

> On Nov 18, 2015, at 5:47 PM, Tom Curry <thomasrcurry at gmail.com> wrote:
> 
> On Wed, Nov 18, 2015 at 4:12 PM, Dan Langille <dan at langille.org> wrote:
> 
>> 
>>> On Nov 17, 2015, at 9:31 PM, Marcelo Araujo <araujobsdport at gmail.com>
>> wrote:
>>> 
>>> 2015-11-18 10:23 GMT+08:00 Dan Langille <dan at langille.org>:
>>> 
>>>> 
>>>>> On Nov 17, 2015, at 8:46 PM, Marcelo Araujo <araujobsdport at gmail.com>
>>>> wrote:
>>>>> 
>>>>> 2015-11-18 3:14 GMT+08:00 Dan Langille <dan at langille.org>:
>>>>> 
>>>>>> 
>>>>>> On Nov 12, 2015, at 1:30 AM, Marcelo Araujo <araujobsdport at gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 2015-11-12 6:34 GMT+08:00 Dan Langille <dan at langille.org>:
>>>>>> 
>>>>>>> On Oct 12, 2015, at 1:00 PM, Dan Langille <dan at langille.org> wrote:
>>>>>>>> 
>>>>>>>> Following up on the discussions during EuroBSDCon 2015 (Stockholm)
>>>>>>> during the FreeBSD Developer
>>>>>>>> Summit regarding various ZFS configuration settings, I write to
>> start
>>>>>>> our implementation phase now that some
>>>>>>>> usual suspects have joined the list.
>>>>>>>> 
>>>>>>>> re https://wiki.freebsd.org/201510DevSummit/Performance
>>>>>>>> 
>>>>>>>> I think the first order of business is granting access rights to the
>>>>>>> server (varm) in question:
>>>>>>>> 
>>>>>>>> http://dan.langille.org/2015/07/19/varm/
>>>>>>>> 
>>>>>>>> During the workshop, mention was made of serial access.  I can
>> arrange
>>>>>>> that.
>>>>>>>> 
>>>>>>>> The server has IPMI, however, my first thought:
>>>>>>>> 
>>>>>>>> 1 - connect a USB-serial cable to varm & link that to another server
>>>> in
>>>>>>> my rack.
>>>>>>> 
>>>>>>> Marcelo: At EuroBSDCon, was it you who mentioned a particular
>>>>>>> configuration for the test machine which made
>>>>>>> it easy to configure and run tests?  Was it PXE booting or something?
>>>>>>> 
>>>>>>>> 2 - create a jail in that server and give it access to that serial
>>>>>>> connection
>>>>>>>> 3 - redirect incoming port XYZ to that jail via a public-key-only
>> ssh
>>>>>>> connection
>>>>>>>> 4 - give people access
>>>>>>>> 
>>>>>>>> Any suggestions?
>>>>>>> 
>>>>>>> —
>>>>>>> Dan Langille
>>>>>>> http://langille.org/
>>>>>>> 
>>>>>>> 
>>>>>> Hello Dan,
>>>>>> 
>>>>>> Yes, was me :)
>>>>>> 
>>>>>> I mention about zopkio test framework.
>>>>>> I gave a presentation last weekend at PyCon Hong Kong about it.
>>>>>> 
>>>>>> Here is my slides:
>>>>>> 
>>>> 
>> http://www.slideshare.net/araujobsd/functional-and-scale-performance-tests-using-zopkio
>>>>>> 
>>>>>> The good of Zopkio is, we can write tests at once and run it as much
>> as
>>>> we
>>>>>> want in different machines. Also Zopkio depends of Naarad, that can
>>>> parse a
>>>>>> CSV file and create metrics and SLA over those metrics, plot graphs
>> and
>>>> so
>>>>>> on. Pretty nice tool!!!
>>>>>> 
>>>>>> I'm wondering if we could start to test something and maybe show it at
>>>>>> AsiaBSDCon and BSDCon(Canada) next year? What do you think?
>>>>>> What I need right now would be a list of tests that we want to perform
>>>> as
>>>>>> well as what parameters we would like to take as metrics to compare.\
>>>>>> 
>>>>>> 
>>>>>> For tests, we can start with this list:
>>>>>> https://github.com/dlangille/zfs_benchmarks/issues
>>>>>> 
>>>>>> We can start as soon as I figure out how to provide access to the
>>>>>> testers.  See above re serial connection.
>>>>>> 
>>>>>> I want to provide access, but I want to keep access restricted to only
>>>>>> this box and not to the rest of my home LAN.  I plan to do this via a
>>>>>> VLAN.
>>>>>> 
>>>>>> I could fire up a Rasperberry Pi and allow ssh into that.  Will that
>> be
>>>>>> enough
>>>>>> power for what you need to do?
>>>>>> 
>>>>>> 
>>>>> First of all, thanks to share the tests cases.
>>>>> 
>>>>> If I use zopkio, the best would be access SSH direct to the target
>>>> machine
>>>>> where I need to run the tests. For zopkio, I need to have my SSH KEY on
>>>> the
>>>>> target machine.
>>>> 
>>>> I am OK with this.
>>>> 
>>>>> As I don't know your network, maybe what you could do is: Via
>>>> RasperBerry,
>>>>> forward the SSH to the target machine, I will pass-through via your
>>>>> RasperBerry where you can control the access for the rest of your LAN.
>>>>> 
>>>>> Another approach could be, two different subnets and a firewall. Or as
>>>> you
>>>>> said, VLANS.
>>>> 
>>>> I will be doing VLANS, which have yet to be set up.
>>>> 
>>>> The target system will have ZFS pools can be configured for different
>>>> tests (i.e. raidz2 vs raidz3).
>>>> This will involve gpart etc because the drives & pools will need to be
>>>> 'wiped' between different test
>>>> runs.
>>>> 
>>>> I seem to recall someone suggesting PXE boot and configuring the system
>>>> remotely.  Does anyone
>>>> recall that?  That aspect of the discussion was not recorded:
>>>> https://wiki.freebsd.org/201510DevSummit/Performance
>>>> 
>>>> 
>>> Bapt@ mentioned that, this is the way how we are doing in another
>> project.
>>> But in my point of view, it is not a must for our case!
>>> 
>>> The PXE wold be good if we try to test different of OS flavors, or build
>>> different images.
>> 
>> OK.  The only thing holding us back is:
>> 
>> - adding the air filters to the case
>> - moving to the new switch with the new VLANs
>> 
>> It's now a matter of time.
>> 
>> —
>> Dan Langille
>> http://langille.org/
>> 
>> 
>> 
>> _______________________________________________
>> freebsd-performance at freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-performance
>> To unsubscribe, send any mail to "
>> freebsd-performance-unsubscribe at freebsd.org"
>> 
> 
> 
> Dan,
> 
> I don't know if you have already cracked the serial access nut, but every
> supermicro motherboard I have owned has supported serial over lan. I
> personally have never used it, I have just noticed it from time to time.
> Here is a good resource on setting it up
> http://serverfault.com/questions/574351/serial-over-lan-on-freebsd-10-0-with-supermicro-x9-scm-f
> (note: it's for an X9 motherboard but it should work for your X10, see the
> comment at the very end)

If we go that way, I will keep that in mind, thank you.

> Also, FreeBSD, ZFS and performance are three of my most favorite words. Is
> there any way I can participate in this adventure?

Yes, there will be, I'm sure. 

This comes to mind immediately: running the tests on your own server will validate/invalidate our results.

For now, we have no results, because it's all waiting on me and my network at home.  It's a time issue.
This weekend is pretty much filled with other stuff.

— 
Dan Langille
http://langille.org/