Re: git: ef2a572bf6bd - main - ipsec_offload: kernel infrastructure

On Fri, Jul 19, 2024 at 04:26:43PM +0200, Kristof Provost wrote:
> On 12 Jul 2024, at 13:25, Konstantin Belousov wrote:
> > The branch main has been updated by kib:
> > 
> > URL: https://cgit.FreeBSD.org/src/commit/?id=ef2a572bf6bdcac97ef29ce631d2f50f938e1ec8
> > 
> > commit ef2a572bf6bdcac97ef29ce631d2f50f938e1ec8
> > Author:     Konstantin Belousov <kib@FreeBSD.org>
> > AuthorDate: 2021-08-22 19:38:04 +0000
> > Commit:     Konstantin Belousov <kib@FreeBSD.org>
> > CommitDate: 2024-07-12 04:27:58 +0000
> > 
> >     ipsec_offload: kernel infrastructure
> > 
> >     Inline IPSEC offload moves almost whole IPSEC processing from the
> >     CPU/MCU and possibly crypto accelerator, to the network card.
> > 
> >     The transmitted packet content is not touched by CPU during TX
> >     operations, kernel only does the required policy and security
> >     association lookups to find out that given flow is offloaded, and
> > then
> >     packet is transmitted as plain text to the card. For driver
> > convenience,
> >     a metadata is attached to the packet identifying SA which must
> > process
> >     the packet. Card does encryption of the payload, padding, calculates
> >     authentication, and does the reformat according to the policy.
> > 
> >     Similarly, on receive, card does the decapsulation, decryption, and
> >     authentification.  Kernel receives the identifier of SA that was
> >     used to process the packet, together with the plain-text packet.
> > 
> >     Overall, payload octets are only read or written by card DMA engine,
> >     removing a lot of memory subsystem overhead, and saving CPU time
> > because
> >     IPSEC algos calculations are avoided.
> > 
> >     If driver declares support for inline IPSEC offload (with the
> >     IFCAP2_IPSEC_OFFLOAD capability set and registering method table
> > struct
> >     if_ipsec_accel_methods), kernel offers the SPD and SAD to driver.
> >     Driver decides which policies and SAs can be offloaded based on
> >     hardware capacity, and acks/nacks each SA for given interface to
> >     kernel.  Kernel needs to keep this information to make a decision to
> >     skip software processing on TX, and to assume processing already
> > done
> >     on RX.  This shadow SPD/SAD database of offloads is rooted from
> >     policies (struct secpolicy accel_ifps, struct ifp_handle_sp) and SAs
> >     (struct secasvar accel_ipfs, struct ifp_handle_sav).
> > 
> >     Some extensions to the PF_KEY socket allow to limit interfaces for
> >     which given SP/SA could be offloaded (proposed for offload).  Also,
> >     additional statistics extensions allow to observe
> > allocation/octet/use
> >     counters for specific SA.
> > 
> >     Since SPs and SAs are typically instantiated in non-sleepable
> > context,
> >     while offloading them into card is expected to require costly async
> >     manipulations of the card state, calls to the driver for offload and
> >     termination are executed in the threaded taskqueue.  It also solves
> >     the issue of allocating resources needed for the offload database.
> >     Neither ipf_handle_sp nor ipf_handle_sav do not add reference to the
> >     owning SP/SA, the offload must be terminated before last reference
> > is
> >     dropped.  ipsec_accel only adds transient references to ensure safe
> >     pointer ownership by taskqueue.
> > 
> >     Maintaining the SA counters for hardware-accelerated packets is the
> >     duty of the driver.  The helper ipsec_accel_drv_sa_lifetime_update()
> >     is provided to hide accel infrastructure from drivers which would
> > use
> >     expected callout to query hardware periodically for updates.
> > 
> >     Reviewed by:    rscheff (transport, stack integration), np
> >     Sponsored by:   NVIDIA networking
> >     Differential revision:  https://reviews.freebsd.org/D44219
> > ---
> >  sys/conf/files               |    2 +
> >  sys/conf/options             |    1 +
> >  sys/modules/ipsec/Makefile   |    5 +-
> >  sys/netipsec/ipsec.c         |   17 +
> >  sys/netipsec/ipsec.h         |   11 +
> >  sys/netipsec/ipsec_input.c   |   11 +
> >  sys/netipsec/ipsec_offload.c | 1061
> > ++++++++++++++++++++++++++++++++++++++++++
> >  sys/netipsec/ipsec_offload.h |  191 ++++++++
> >  sys/netipsec/ipsec_output.c  |   15 +
> >  sys/netipsec/ipsec_pcb.c     |   38 +-
> >  sys/netipsec/key.c           |  270 ++++++++++-
> >  sys/netipsec/key.h           |    6 +
> >  sys/netipsec/key_debug.c     |    5 +
> >  sys/netipsec/keydb.h         |   14 +
> >  14 files changed, 1628 insertions(+), 19 deletions(-)
> > 
> I’m seeing messages like `ipsec_accel_sa_install_newkey: spi 0x1001 flags
> 0x40 seq 0` running the test suite now.
> Are those stray debug printfs?

Not stray, but also they should not be there by default indeed.
https://reviews.freebsd.org/D46045