svn commit: r254859 - stable/9/sys/geom/eli

[John-Mark Gurney]

Ollivier Robert wrote this message on Sun, Aug 25, 2013 at 14:29 +0000:
> Author: roberto
> Date: Sun Aug 25 14:29:47 2013
> New Revision: 254859
> URL: http://svnweb.freebsd.org/changeset/base/254859
> 
> Log:
>   MFC r226840: AESNI-related commit, from a long time ago:
>   
>   Before this change when GELI detected hardware crypto acceleration it will
>   start only one worker thread. For software crypto it will start by default
>   N worker threads where N is the number of available CPUs.
>   
>   This is not optimal if hardware crypto is AES-NI, which uses CPU for AES
>   calculations.
>   
>   Change that to always start one worker thread for every available CPU.
>   Number of worker threads per GELI provider can be easly reduced with
>   kern.geom.eli.threads sysctl/tunable and even for software crypto it
>   should be reduced when using more providers.
>   
>   While here, when number of threads exceeds number of CPUs avilable don't
>   reduce this number, assume the user knows what he is doing.
>   
>   Submitted by:	Michael Moll <kvedulv at kvedulv.de>

I'll just point out that if you have a large number of geli volumes,
that this turns out to hurt performance rather than help it..  At
least in my testing on a 6 core machine w/ an 8 disk zfs pool where
all 8 disks are encrypted...

I think the problem is that we launch threads on a per geli/disk instead
of just having a generic set of threads that can work with any geli
volume...  When you have 17 geli volumes (like I do on my machine) * 6
threads, thats a lot of threads, imagine if the box had 32 or 64 cores?
You're looking at 500-1000 threads just for encryption...

In case someone asks, 8 zfs disks + 2 spares, swap, l2arc * 2, zil * 2
and zfs mirrored root all encrypted volumes...

-- 
  John-Mark Gurney				Voice: +1 415 225 5579

     "All that I will do, has been done, All that I have, has not."