GELI strangeness with gstat

[Pete French]

Tnaks for the reply....


> If you just did a kldload aesni, but did not reattach the geli device,
> then you are still using software encryption...  You should see something
> like this:
> GEOM_ELI: Device gpt/werner.eli created.
> GEOM_ELI: Encryption: AES-XTS 256
> GEOM_ELI:     Crypto: hardware
> 
> if you are using AES-NI...

Yes, thats exactly what I see - I rebooted the system (had to as I was 
enabling AES_NI in the BIOS) and am loading the crypto and aesni modules 
at boot time form loader.conf. I see this in dmesg:

Enter passphrase for ada0p4:
GEOM_ELI: Device ada0p4.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI:     Crypto: hardware
GEOM_ELI: Device ada1p4.eli created.
GEOM_ELI: Encryption: AES-XTS 128
GEOM_ELI:     Crypto: hardware

and 'geli list' says 'Crypto: hardware' in its output

> Also, what version of FreeBSD are you using?  If you're using pre-10.0-R,
> the performance increase from using AES-NI is only marginal...

Am using 11.1-STABLE from mid June - is it worth getting the last few 
weeks of updates ? I wouldnt have thought so as I havent see any crypto 
chnages go past, but I can give it a go...

> The above does make it look like you're disks are CPU bound by the
> encryption...

Indeed. But the odd this is it only started happening *after* I upgraded 
the machine to much faster CPU's. I am suspecting (hoping!) its actually 
a statistical anomaly in gstat and not a real effect.

> To get a better idea of what is happening, you can run top -S to see
> how much CPU the geli threads are using.

Ah, good idea.

Hmmmm... that shows up 12 threads, each using about 0.2% CPU.


> Also, how many eli volumes do you have?  In my case, I have 13...  In
> order to reduce the load on the scheduler, I have:
> kern.geom.eli.threads="1"

This is interesting - my understanding was that with hardware encryption 
that setting did nothing. From the manpage:

kern.geom.eli.threads: 0
  Specifies how many kernel threads should be used for doing software 
cryptography. Its purpose is to increase performance on SMP systems. If 
hardware acceleration is available, only one thread will be started. If 
set to 0, CPU-bound thread will be started for every active CPU.

So my expectation would be that I only get 1 thread - but actuyally I 
see 12 in 'top -S'. Again, is that a bug or out of date documentation ?

I only have two volumes, and am runnign ZFS over the top with 
compresison enabled, which I think only uses a single write thread, so 
maybe reducing that would help. But theres still a very large 
discrepancy between what all of my other jetricsare showing me and gstat.

Te underlying discs are SSD's by the way, so I am fairly sure thats not 
a bottleneck (and gstats dosnt show the undrelying drive busy).

-pete.