Getting started with ktls

[John Baldwin]

On 3/18/21 8:31 AM, tech-lists wrote:
> On Wed, Mar 17, 2021 at 08:39:02PM +0000, Rick Macklem wrote:
> 
>> Make sure you've done the following:
>> ktls_ocf - is loaded
>> these sysctls are set to 1
>> kern.ipc.tls.enable
>> kern.ipc.mb_use_ext_pgs
> 
> [on stable/13]
> 
> % sysctl kern.ipc.tls.enable kern.ipc.mb_use_ext_pgs
> kern.ipc.tls.enable: 1
> kern.ipc.mb_use_ext_pgs: 1
> 
> % kldstat | grep ktls
> 7    1 0xffff000001353000    25520 ktls_ocf.ko
> %
> 
> % sysctl -a | fgrep kern.ipc.tls.stats
> kern.ipc.tls.stats.ocf.retries: 0
> kern.ipc.tls.stats.ocf.separate_output: 0
> kern.ipc.tls.stats.ocf.inplace: 0
> kern.ipc.tls.stats.ocf.tls13_gcm_crypts: 0
> kern.ipc.tls.stats.ocf.tls12_gcm_crypts: 0
> kern.ipc.tls.stats.ocf.tls11_cbc_crypts: 0
> kern.ipc.tls.stats.ocf.tls10_cbc_crypts: 0
> kern.ipc.tls.stats.switch_failed: 0
> kern.ipc.tls.stats.switch_to_sw: 0
> kern.ipc.tls.stats.switch_to_ifnet: 0
> kern.ipc.tls.stats.failed_crypto: 0
> kern.ipc.tls.stats.corrupted_records: 0
> kern.ipc.tls.stats.active: 0
> kern.ipc.tls.stats.enable_calls: 535
> kern.ipc.tls.stats.offload_total: 0
> kern.ipc.tls.stats.sw_rx_inqueue: 0
> kern.ipc.tls.stats.sw_tx_inqueue: 0
> kern.ipc.tls.stats.threads: 4
> %

FYI, you can do this a bit more efficiently with just 'sysctl kern.ipc.tls.stats'

The 'enable_calls' means that OpenSSL is trying to offload connections,
but those attempts are all failing (offload_total is a count of how many
of those setsockopt() calls succeed).  If you are familiar with dtrace,
you can use some DTrace probes to see why 'ktls_enable_tx' and 'ktls_enable_rx'
are failing, or barring that printf.  For example, does ktls_create_session()
fail, or does ktls_try_sw() fail?  It is probably easiest to debug this using
a userland application using openssl than trying NFS over TLS.

-- 
John Baldwin