ZFS/compression/performance

[Radio młodych bandytów]

On 2011-10-13 14:00, freebsd-fs-request at freebsd.org wrote:
> An option is not too compress with ZFS rather directly with gzip however I
> would still need lots of temporary storage for manipulation, which is what
> I am doing now (e.g., sort). Processing with zcat isn't always a good
> solution because some applications want files, but you have to do what you
> have to do.
It seems that with your data gzipping directly is a better option. 
Though I suggest that you experiment with codecs that support larger 
dictionary, i.e. 7zip, I expect that you would see huge strength 
improvement with something like 7z a -mx=1 -md=26 out.7z in. You can use 
higher -md values if you have enough memory, compression mode 1 (mx=1) 
uses 4,5*2^md bytes of RAM, so if my maths is good, md=26 uses ~288 MB. 
If LZMA is too slow, you can at least try 7-zip's deflate64. It's not 
great, but not as bad as gzip.

-- 
Twoje radio