recommended memory for zfs

[Ronald Klop]

On Fri, 10 May 2013 04:18:42 +0200, Benjamin Adams  
<benjamindadams at gmail.com> wrote:

> On 05/09/2013 10:06 PM, Jeremy Chadwick wrote:
>> On Thu, May 09, 2013 at 09:47:27PM -0400, Benjamin Adams wrote:
>>> On 05/09/2013 08:53 PM, Shane Ambler wrote:
>>>> On 09/05/2013 22:48, Benjamin Adams wrote:
>>>>> Hello zfs question about memory.
>>>>> I heard zfs is very ram hungry.
>>>>> Service looking to run:
>>>>> - nginx
>>>>> - postgres
>>>>> - php-fpm
>>>>> - python
>>>>>
>>>>> I have a machine with two quad core cpus but only 4 G Memory
>>>>>
>>>>> I'm looking to buy more ram now.
>>>>> What would be the recommend amount of memory for zfs across 6 drives  
>>>>> on
>>>>> this setup?
>>>>>
>>>> I believe I heard a calculation of 1GB cache per 1TB of disk. But
>>>> basically zfs will use all free ram available if you access that
>>>> much data from disk. You will want to set vfs.zfs.arc_max to allow
>>>> enough ram for your apps to work in.
>>>>
>>>> If you consider the files for your website and the data you store
>>>> you may find that you would never fill more than 500MB of cache.
>>>>
>>>> If you will be serving large media files that will easily use up
>>>> the cache you could give them their own filesystem that only
>>>> caches metadata - zfs set primarycache=metadata zroot/mediafiles
>>>>
>>>>
>>> Thanks for all the replies  Size of DB and HD's are:
>>>
>>> Current DB Size = 23 GB
>>> HD sizes = (6) 500 GB drives
>> Nobody is going to be able to give you a precise/accurate recommendation
>> given the lack of detail provided, I'm sorry to say.  What's the RES
>> size of nginx (all processes combined)?  What's the RES size of
>> postgres (same)?  Do you have PHP scripts that "run amok" for long
>> periods of time and take up lots of RAM?  Same with python?  How many
>> concurrent visitors and what sort of content are you hosting?  Do you
>> maintain/write your own PHP/Python code or are you using some crap like
>> Wordpress?
>>
>> This is just a **small** list of questions -- and what may come as a
>> shock is that I do not expect you to provide answers to any of them.
>> They are questions that you should, for yourself, attempt to answer and
>> work out what you need from there ("teach a man to fish" and all that).
>>
>> The advice of "1GB of RAM per 1TB of disk space" is absolute nonsense on
>> numerous levels -- whoever gave this advice to Shane either has no
>> understanding of how filesystems/ZFS works, or does but chose to
>> simplify to the point where they're providing half-ass information.
>> There is no direct, or even indirect, correlation between disk capacity
>> and ZFS ARC size -- what matter is your "working set" (to quote Tom).
>> You need to have some idea of how much disk I/O you're doing, and what
>> type of I/O (sequential or random).
>>
>> If you want my general advice, Benjamin, it's this: get yourself a
>> system with *minimum* 8GB of RAM but has the physical possibility of
>> supporting more (and only add more RAM when/if you know you need it); do
>> not bother with ZFS on a system with 4GB.  Run amd64, not i386 (I don't
>> recommend bothering with ZFS on i386 -- I am not going to get into a
>> discussion about this either).  Run stable/9, not 9.1-RELEASE.  Avoid
>> compression and dedup.  And test disk failures as well (don't get caught
>> with your pants down later).
>>
>> The above advice comes from someone who did hosting (web/ssh/etc.) for
>> almost 20 years with KISS principle applied at all levels.  YMMV though,
>> depending on what all you're doing/what you truly need.
>>
>> Good luck.
>>
> Jeremy,
>
> Was just see if I should just get raid controller and more ram down the  
> road.
> List of priorities.
>
> Main thing is I move from BSD when 9.0 came out.  Was looking to see if  
> zfs is included in the installer.  Now.
>
> Sum up:
> upgrade ram to 16GB (not 64 like plained)
> and  raid controller that supports level 5.
>

Let ZFS do the RAID stuff. Do not use a RAID controller, but give the  
plain disks to ZFS. Some of the nice features come from ZFS doing the RAID  
stuff.

Ronald.