Re: panic in cypto code

On Thu, Oct 05, 2023 at 06:05:37PM +0200, Kristof Provost wrote:
> Hi Steve,
> 
> On 5 Oct 2023, at 17:36, Steve Kargl wrote:
> > In case anyone else is using openvpn.
> >
> > %  pkg info openvpn
> > openvpn-2.6.6
> > Name           : openvpn
> > Version        : 2.6.6
> > Installed on   : Tue Sep 19 08:48:55 2023 PDT
> > Origin         : security/openvpn
> > Architecture   : FreeBSD:15:amd64
> >
> > % uname -a
> > FreeBSD hotrats 15.0-CURRENT #1 main-n265325-9c30461dd25b:
> > Thu Sep 14 08:09:18 PDT 2023 kargl@hotrats:$PATH/HOTRATS amd64
> >
> >
> > Fatal double fault
> > rip 0xffffffff8099b408 rsp 0xfffffe000e1cc000 rbp 0xfffffe000e1cc010
> > rax 0x53749f62934c5349 rdx 0x53749f62934c5349 rbx 0xfffffe000e1cc200
> > rcx 0x57bf32fec3cbde70 rsi 0x32e8db2f0591c5da rdi 0x832f0fb1e6d07eb0
> > r8 0 r9 0 r10 0
> > r11 0x60 r12 0x5af7589946bd13d9 r13 0xbeddd6a808e1dd54
> > r14 0xcdf12bbf2708189c r15 0xeb262ae8536a7adf rflags 0x10246
> > cs 0x20 ss 0x28 ds 0x3b es 0x3b fs 0x13 gs 0x1b
> > fsbase 0x1c02e381d120 gsbase 0xffffffff81a10000 kgsbase 0
> > cpuid = 0; apic id = 00
> > panic: double fault
> > cpuid = 0
> > time = 1696512769
> > KDB: stack backtrace:
> > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffffff812b3060
> > vpanic() at vpanic+0x132/frame 0xffffffff812b3190
> > panic() at panic+0x43/frame 0xffffffff812b31f0
> > dblfault_handler() at dblfault_handler+0x1ce/frame 0xffffffff812b32b0
> > Xdblfault() at Xdblfault+0xd7/frame 0xffffffff812b32b0
> > --- trap 0x17, rip = 0xffffffff8099b408, rsp = 0xfffffe000e1cc000, rbp = 0xfffffe000e1cc010 ---
> > gfmultword4() at gfmultword4+0x8/frame 0xfffffe000e1cc010
> > gf128_mul4b() at gf128_mul4b+0x63/frame 0xfffffe000e1cc050
> > AES_GMAC_Update() at AES_GMAC_Update+0x73/frame 0xfffffe000e1cc0b0
> > swcr_gcm() at swcr_gcm+0x660/frame 0xfffffe000e1cc830
> > swcr_process() at swcr_process+0x1a/frame 0xfffffe000e1cc850
> > crypto_dispatch() at crypto_dispatch+0x42/frame 0xfffffe000e1cc870
> > ovpn_transmit_to_peer() at ovpn_transmit_to_peer+0x54e/frame 0xfffffe000e1cc8d0
> > ovpn_output() at ovpn_output+0x2a2/frame 0xfffffe000e1cc950
> > ip_output() at ip_output+0x11f6/frame 0xfffffe000e1cca40
> > ovpn_encap() at ovpn_encap+0x3e7/frame 0xfffffe000e1ccac0
> >
> > #13 0xffffffff80ae08ce in dblfault_handler (frame=<optimized out>)
> >     at /usr/src/sys/amd64/amd64/trap.c:1012
> > #14 <signal handler called>
> > #15 0xffffffff8099b408 in gfmultword4 (worda=3668422891496654298,
> >     wordb=9452791399630012080, wordc=6013606648173318985,
> >     wordd=6322828471639465584, x=..., tbl=0xfffffe000e1cc200)
> >     at /usr/src/sys/opencrypto/gfmult.c:174
> > #16 0xffffffff8099b5d3 in gf128_mul4b (r=...,
> >     v=v@entry=0xfffff800076b9a64 "\3156}\373\312w\254iBnD\001ܹ˾\353&*\350Sjz߃/\017\261\346\320~\260Z\367X\231F\275\023\331St\237b\223LSI\276\335֨\b\341\335TW\2772\376\303\313\336pN\265\023\352\2054\002\a/˦9R\321\366p\f\352\204P\360\270\371\250\\\aE?7s\377\253\217b\262%\214\317m",
> >     tbl=tbl@entry=0xfffffe000e1cc200) at /usr/src/sys/opencrypto/gfmult.c:268
> > #17 0xffffffff8099ab13 in AES_GMAC_Update (ctx=0xfffffe000e1cc200,
> >     vdata=<optimized out>, len=144) at /usr/src/sys/opencrypto/gmac.c:94
> > #18 0xffffffff80998ae0 in swcr_gcm (ses=0xfffff8020376a048,
> >     crp=0xfffff80023386c08) at /usr/src/sys/opencrypto/cryptosoft.c:505
> > #19 0xffffffff80997c4a in swcr_process (dev=<optimized out>,
> >     crp=0xfffff80023386c08, hint=<optimized out>)
> >     at /usr/src/sys/opencrypto/cryptosoft.c:1680
> >
> Do you have a bit more information about what happened here?
> As in: can you reproduce this, or do you have any idea what
> was going on to trigger this?  Did anything change in your
> setup (i.e. is if_ovpn use new, or did you update either kernel
> or userspace or ?

I updated the system on the date displayed by 'uname -a'.
This included both base system and all installed ports;
including openvpn.  I normally leave the system running Xorg,
and I would find the system in a "locked-up" blank-screen
saver state.  I assumed I was having a Xorg/drm-kmod problem,
so I shut Xorg down last night.  The above panic was waiting
for me this morning.  The panic happens every night.

Note , I don't use if_ovpn.  This a client over a tun0 device
through wlan0.

> 
> Do you have the full core dump to poke at?
> 

Yes, I do, but it's on a home system.  I can put it up on
my kargl@freefall later tonight (in 10-ish hours).  I'll
include the dmesg.boot so you have some idea about the 
hardware.

> It might be a bug in the crypto code, but it could also
> be a bug in the if_ovpn code, so I’d like to work out
> what caused this.

Thanks for the quick reply.

-- 
Steve