Re: panic in cypto code

On 5 Oct 2023, at 19:34, Steve Kargl wrote:
> On Thu, Oct 05, 2023 at 06:05:37PM +0200, Kristof Provost wrote:
>> Hi Steve,
>>
>> On 5 Oct 2023, at 17:36, Steve Kargl wrote:
>>> In case anyone else is using openvpn.
>>>
>>> %  pkg info openvpn
>>> openvpn-2.6.6
>>> Name           : openvpn
>>> Version        : 2.6.6
>>> Installed on   : Tue Sep 19 08:48:55 2023 PDT
>>> Origin         : security/openvpn
>>> Architecture   : FreeBSD:15:amd64
>>>
>>> % uname -a
>>> FreeBSD hotrats 15.0-CURRENT #1 main-n265325-9c30461dd25b:
>>> Thu Sep 14 08:09:18 PDT 2023 kargl@hotrats:$PATH/HOTRATS amd64
>>>
>>>
>>> Fatal double fault
>>> rip 0xffffffff8099b408 rsp 0xfffffe000e1cc000 rbp 0xfffffe000e1cc010
>>> rax 0x53749f62934c5349 rdx 0x53749f62934c5349 rbx 0xfffffe000e1cc200
>>> rcx 0x57bf32fec3cbde70 rsi 0x32e8db2f0591c5da rdi 0x832f0fb1e6d07eb0
>>> r8 0 r9 0 r10 0
>>> r11 0x60 r12 0x5af7589946bd13d9 r13 0xbeddd6a808e1dd54
>>> r14 0xcdf12bbf2708189c r15 0xeb262ae8536a7adf rflags 0x10246
>>> cs 0x20 ss 0x28 ds 0x3b es 0x3b fs 0x13 gs 0x1b
>>> fsbase 0x1c02e381d120 gsbase 0xffffffff81a10000 kgsbase 0
>>> cpuid = 0; apic id = 00
>>> panic: double fault
>>> cpuid = 0
>>> time = 1696512769
>>> KDB: stack backtrace:
>>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffffff812b3060
>>> vpanic() at vpanic+0x132/frame 0xffffffff812b3190
>>> panic() at panic+0x43/frame 0xffffffff812b31f0
>>> dblfault_handler() at dblfault_handler+0x1ce/frame 0xffffffff812b32b0
>>> Xdblfault() at Xdblfault+0xd7/frame 0xffffffff812b32b0
>>> --- trap 0x17, rip = 0xffffffff8099b408, rsp = 0xfffffe000e1cc000, rbp = 0xfffffe000e1cc010 ---
>>> gfmultword4() at gfmultword4+0x8/frame 0xfffffe000e1cc010
>>> gf128_mul4b() at gf128_mul4b+0x63/frame 0xfffffe000e1cc050
>>> AES_GMAC_Update() at AES_GMAC_Update+0x73/frame 0xfffffe000e1cc0b0
>>> swcr_gcm() at swcr_gcm+0x660/frame 0xfffffe000e1cc830
>>> swcr_process() at swcr_process+0x1a/frame 0xfffffe000e1cc850
>>> crypto_dispatch() at crypto_dispatch+0x42/frame 0xfffffe000e1cc870
>>> ovpn_transmit_to_peer() at ovpn_transmit_to_peer+0x54e/frame 0xfffffe000e1cc8d0
>>> ovpn_output() at ovpn_output+0x2a2/frame 0xfffffe000e1cc950
>>> ip_output() at ip_output+0x11f6/frame 0xfffffe000e1cca40
>>> ovpn_encap() at ovpn_encap+0x3e7/frame 0xfffffe000e1ccac0
>>>
>>> #13 0xffffffff80ae08ce in dblfault_handler (frame=<optimized out>)
>>>     at /usr/src/sys/amd64/amd64/trap.c:1012
>>> #14 <signal handler called>
>>> #15 0xffffffff8099b408 in gfmultword4 (worda=3668422891496654298,
>>>     wordb=9452791399630012080, wordc=6013606648173318985,
>>>     wordd=6322828471639465584, x=..., tbl=0xfffffe000e1cc200)
>>>     at /usr/src/sys/opencrypto/gfmult.c:174
>>> #16 0xffffffff8099b5d3 in gf128_mul4b (r=...,
>>>     v=v@entry=0xfffff800076b9a64 "\3156}\373\312w\254iBnD\001ܹ˾\353&*\350Sjz߃/\017\261\346\320~\260Z\367X\231F\275\023\331St\237b\223LSI\276\335֨\b\341\335TW\2772\376\303\313\336pN\265\023\352\2054\002\a/˦9R\321\366p\f\352\204P\360\270\371\250\\\aE?7s\377\253\217b\262%\214\317m",
>>>     tbl=tbl@entry=0xfffffe000e1cc200) at /usr/src/sys/opencrypto/gfmult.c:268
>>> #17 0xffffffff8099ab13 in AES_GMAC_Update (ctx=0xfffffe000e1cc200,
>>>     vdata=<optimized out>, len=144) at /usr/src/sys/opencrypto/gmac.c:94
>>> #18 0xffffffff80998ae0 in swcr_gcm (ses=0xfffff8020376a048,
>>>     crp=0xfffff80023386c08) at /usr/src/sys/opencrypto/cryptosoft.c:505
>>> #19 0xffffffff80997c4a in swcr_process (dev=<optimized out>,
>>>     crp=0xfffff80023386c08, hint=<optimized out>)
>>>     at /usr/src/sys/opencrypto/cryptosoft.c:1680
>>>
>> Do you have a bit more information about what happened here?
>> As in: can you reproduce this, or do you have any idea what
>> was going on to trigger this?  Did anything change in your
>> setup (i.e. is if_ovpn use new, or did you update either kernel
>> or userspace or ?
>
> I updated the system on the date displayed by 'uname -a'.
> This included both base system and all installed ports;
> including openvpn.  I normally leave the system running Xorg,
> and I would find the system in a "locked-up" blank-screen
> saver state.  I assumed I was having a Xorg/drm-kmod problem,
> so I shut Xorg down last night.  The above panic was waiting
> for me this morning.  The panic happens every night.
>
> Note , I don't use if_ovpn.  This a client over a tun0 device
> through wlan0.
>
The backtrace contradicts you, but DCO is relatively transparent, so it’s quite possible you didn’t notice. It defaults to being enabled, and ought to just work.

>>
>> Do you have the full core dump to poke at?
>>
>
> Yes, I do, but it's on a home system.  I can put it up on
> my kargl@freefall later tonight (in 10-ish hours).  I'll
> include the dmesg.boot so you have some idea about the
> hardware.
>
That’d be very helpful, thanks.

Best regards,
Kristof