Hardware RAID or software for ZFS

[bsd at todoo.biz]

> Le 18 nov. 2014 à 18:29, Matthew Seaman <matthew at freebsd.org> a écrit :
> 
> On 11/18/14 15:28, bsd at todoo.biz wrote:
>> I would need your help in order to figure out which will be the best in order to setup a server which will be used as a Poudriere server. 
>> 
>> My question is related to the usage of hardware RAID controller (JBOD) or software based controller (graid) 
>> 
>> Server we are targeting is a SuperMicro 1018R-WC0R - 1U
>> 
>> I used to build my server using an HBA such as LSI SAS 9207-8i or LSI SAS 9201-16i
>> Then I’ll configure the HBA as JBOD so that disks can be presented correctly to the system so that we can build our ZFS FS on top of that. 
>> 
>> 
>> My sysadmin suggested me to get rid of the HBA and use direct connectivity and Graid. 
>> 
>> 
>> I would like to have some feedback on various aspects : 
>> 
>> 1. performance 
>> 2. usability 
>> 3. problem one might have encountered
>> 
> 
> OK -- Your sysadmin is correct about the direct connectivity thing, but
> graid?  Seriously?  That's an option for really cheap and some might say
> quite nasty pseudo-raid things you get on some desktop hardware.  If
> you're going to be building a significant number of packages with any
> regularity, then you're going to need something better than that.
> 
> The LSI HBAs you suggest are known to work well with FreeBSD.  If you're
> going to be using ZFS -- which is really a no-brainer for a poudriere
> box, as poudriere has a lot of functionality built around cloning and
> snapshotting and other things that come naturally to ZFS[*] -- well, in
> that case, ideally you want ZFS to have direct access to the disk devices.
> 
> With LSI, when they say 'JBOD' this direct access is generally what they
> mean -- although depending on the precise model you get, you may need to
> flash the device with different firmware to get the characteristics you
> want.  Some other RAID controller manufacturers essentially give you a
> load of single-disk RAID-0 devices, so there's still a layer of RAID
> controller stuff between ZFS and the disk.  That isn't the kiss of death
> to the idea of building a zpool from it, but it's less optimal.
> 
> On the three aspecs you mention:
> 
>   performance depends very much on the precise type of disk hardware
> you specify, how you configure the zpool (for instance, will you be
> using ZIL or ARC devices?) and on having enough RAM in the system to
> provide an effective buffer-cache for all those ZFS filesystems.
> 
>   usability.  I'd say ZFS is pretty usable.  Once it's set up, you
> don't really need to interact with things at the level of the HBA.  It
> just works.
> 
>   problems: this depends very much on the details of what you're trying
> to do.  But IMHO you're on the right track with what you've described so
> far.
> 
> 	Cheers,
> 
> 	Matthew
> 
> [*] Although poudriere does now work on UFS as well and has done for a
> year or so.
> 

Thank you very much for this documented and precise answer. 

I think that the hardware we were targeting is a bit too new and is not yet marked as « ok » for FreeBSD by SuperMicro. 
We will probably go for a little bit older hardware that’s 100% known to work ok with FreeBSD. 

I have already been building quite large NAS using this setup (JBOD Controller + ZFS on top) and I had very interesting perfs. 
I didn’t know about the problem with native JBOD and controller SW update/upgrade…  

> SuperMicro 1018R-WC0R - 1U


This is the one we wanted to target but It doesn’t seem to be on the hardware list marked as FreeBSD compatible. 


Thanks all - long leave FBSD. 

;-) 

> 
> 
> 
> 

________________________________________________
«?»¥«?»§«?»¥«?»§«?»¥«?»§«?»¥«?»§«?»¥«?»§«?»¥«?»§
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
BSD - BSD - BSD - BSD - BSD - BSD - BSD - BSD -
________________________________________________
«?»¥«?»§«?»¥«?»§«?»¥«?»§«?»¥«?»§«?»¥«?»§«?»¥«?»§
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
PGP ID --> 0x1BA3C2FD