FreeBSD, Centos and ZFS

[aurfalien]

On Oct 15, 2013, at 3:08 AM, Johan Hendriks wrote:

> aurfalien wrote:
>> On Oct 12, 2013, at 10:56 AM, Mark Felder wrote:
>> 
>>> On Sat, Oct 12, 2013, at 10:53, aurfalien wrote:
>>>> Hi,
>>>> 
>>>> I would like to first say that by no means is this a "hey, why is my Mac
>>>> faster then my PC" kind of email.
>>>> 
>>>> I'm really hoping its an LSI driver issue.
>>>> 
>>> It may very well be an LSI firmware issue. What are the firmwares for
>>> those HBAs?
>> Upon doing this;
>> 
>> sysctl -a | grep mps
>> 
>> I get this;
>> 
>> dev.mps.0.driver_version: 14.00.00.01-fbsd
>> 
>> LSIs site mentions the latest drives at being 17.00.00.00
>> 
>> I'll go ahead and install the latest to see what happens.
>> 
>> Whats the best way to do this, I assume build it and load via loader.conf?
>> 
>> - aurf
>> _______________________________________________
>> freebsd-questions at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
>> To unsubscribe, send any mail to "freebsd-questions-unsubscribe at freebsd.org"
> The way i do it is from FreeBSD itself.
> I did it on FreeBSD 9.0 and 9.2, never did it on a 9.1 or 10 Current. I also never did it on cards other then the 9211-8i LSI cards. 
> So do not hold me responsible if things go wrong :D
> If things go wrong it can damage the card so it has to be put in for RMA
> Download the firmware, and the sas2flash file for FreeBSD 
> 
> Then i do the following
>  
> 
> myrootonzfs ~ # ./sas2flash -listall
> 
> 
Rather then up the driver in FreeBSD 9, (I tried and it didn't go too well anyways) I left it at v14 and down graded the firmware.  While the Intel rebranded LSI controllers were stuck at an older Intel version, I decided to make them all v14 of the LSI firmware.

Throughput has improved a bit and it seems more consistent in terms of test results.

So the key here in FreeBSD is to match firmware with driver versions.  Linux didn't seem to care but all OS's have there nuances so I'm not concerned.

In terms of flashing the firmware, I had to do it via EFI using the -force flag.  Even though v14 didn't have an EFI tool , v17 did so I used it while specifying v14 firmware file.

But a note on the Linux comparison;

While I mimicked my values on both platforms, I did not mimic all values.

ZoL comes with a little interesting feature; arc2 compression, which does seem to enhance performance all around.  I disabled this in CentOS to level the playing field.

There are several other things I saw on ZoL which seem very new and more in line with ZFS on FreeBSD 10.

So I installed FreeBSD 10, set up my prefs as before but performance were very close to FreeBSD 9, which were both lower to ZoL in CentOS.

I've attached my IOzone results in zipped xls format if any one is interested.  I'm still baffled as to why CentOS has substantially better results.

* Concerning the graphs;
Scales are all set equal to allow for easy comparisons.
Values to the right are avg file sizes in my env;

8MB
12MB
16MB
30MB
53MB
2GB

At any rate, I'm hoping some one could help solve this performance discrepancy.

I suppose a dump of my hardware list and ZFS properties are in order but only if any one is interested as I don't like typing in vein :)

Thanks in advance,

- aurf