[Bug 264273] FreeBSD 13 nginx kernel panic probably sendfile

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=264273

            Bug ID: 264273
           Summary: FreeBSD 13 nginx kernel panic probably sendfile
           Product: Base System
           Version: 13.0-RELEASE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: bblister@gmail.com

I performed a binary upgrade from 12 to 13 Before 8 hours. I went to sleep, and
in the morning I saw 2 kernel panics have occurred. 

Both traces indicate that all started from nginx and especially vn_sendfile.

nginx is installed from binary packages.

This is a production server running a couple of websites, inside a cloud VPS. 
It was running 12.2-RELEASE-p14 rock solid without any reboot, without any
panic. The configuration is not changed.

I issued now a  sysctl net.inet.tcp.tso=0 
to see if it will make the server more reliable.
If kernel still panics I will have to disable sendfile in nginx.

I cannot switch to stable and perform kernel patches, because this is in
production. I can issue commands or change configuration or display specific
config files. In



>uname -a
FreeBSD arch.ece.uowm.gr 13.0-RELEASE-p11 FreeBSD 13.0-RELEASE-p11 #0: Tue Apr 
5 18:54:35 UTC 2022    
root@amd64-builder.daemonology.net:/usr/obj/usr/src/amd64.amd64/sys/GENERIC 
amd64


Kernel panics:

May 27 04:13:49 arch kernel: Fatal trap 12: page fault while in kernel mode
May 27 04:13:49 arch kernel: cpuid = 2; apic id = 02
May 27 04:13:49 arch kernel: fault virtual address      = 0x58
May 27 04:13:49 arch kernel: fault code         = supervisor read data, page
not present
May 27 04:13:49 arch kernel: instruction pointer        =
0x20:0xffffffff81086c80
May 27 04:13:49 arch kernel: stack pointer              =
0x28:0xfffffe0073dedf50
May 27 04:13:49 arch kernel: frame pointer              =
0x28:0xfffffe0073dedf50
May 27 04:13:49 arch kernel: code segment               = base 0x0, limit
0xfffff, type 0x1b
May 27 04:13:49 arch kernel:                    = DPL 0, pres 1, long 1, def32
0, gran 1
May 27 04:13:49 arch kernel: processor eflags   = interrupt enabled, resume,
IOPL = 0
May 27 04:13:49 arch kernel: current process            = 65527 (nginx)
May 27 04:13:49 arch kernel: trap number                = 12
May 27 04:13:49 arch kernel: panic: page fault
May 27 04:13:49 arch kernel: cpuid = 2
May 27 04:13:49 arch kernel: time = 1653612365
May 27 04:13:49 arch kernel: KDB: stack backtrace:
May 27 04:13:49 arch kernel: #0 0xffffffff80c57535 at kdb_backtrace+0x65
May 27 04:13:49 arch kernel: #1 0xffffffff80c09f11 at vpanic+0x181
May 27 04:13:49 arch kernel: #2 0xffffffff80c09d83 at panic+0x43
May 27 04:13:49 arch kernel: #3 0xffffffff8108b1a7 at trap_fatal+0x387
May 27 04:13:49 arch kernel: #4 0xffffffff8108b1ff at trap_pfault+0x4f
May 27 04:13:49 arch kernel: #5 0xffffffff8108a85d at trap+0x27d
May 27 04:13:49 arch kernel: #6 0xffffffff81061f08 at calltrap+0x8
May 27 04:13:49 arch kernel: #7 0xffffffff80c9c38f at m_pullup+0x1af
May 27 04:13:49 arch kernel: #8 0xffffffff821162bf at ipfw_chk+0x3fcf
May 27 04:13:49 arch kernel: #9 0xffffffff8211805c at ipfw_check_frame+0x13c
May 27 04:13:49 arch kernel: #10 0xffffffff80d422c7 at pfil_run_hooks+0x97
May 27 04:13:49 arch kernel: #11 0xffffffff80d23bf4 at ether_output_frame+0x94
May 27 04:13:49 arch kernel: #12 0xffffffff80d23b08 at ether_output+0x6b8
May 27 04:13:49 arch kernel: #13 0xffffffff80db3ca5 at ip_output_send+0x75
May 27 04:13:49 arch kernel: #14 0xffffffff80db3ac2 at ip_output+0x12b2
May 27 04:13:49 arch kernel: #15 0xffffffff80dc9ab4 at tcp_output+0x1b04
May 27 04:13:49 arch kernel: #16 0xffffffff80ddb189 at tcp_usr_send+0x229
May 27 04:13:49 arch kernel: #17 0xffffffff80c07e2a at vn_sendfile+0x197a
May 27 04:13:49 arch kernel: Uptime: 1h42m58s







--------------------Another Panic--------------------

May 27 05:09:25 arch kernel: Fatal trap 12: page fault while in kernel mode
May 27 05:09:25 arch kernel: cpuid = 0; apic id = 00
May 27 05:09:25 arch kernel: fault virtual address      = 0x580
May 27 05:09:25 arch kernel: fault code         = supervisor read data, page
not present
May 27 05:09:25 arch kernel: instruction pointer        =
0x20:0xffffffff81086c80
May 27 05:09:25 arch kernel: stack pointer              =
0x28:0xfffffe00bb153f50
May 27 05:09:25 arch kernel: frame pointer              =
0x28:0xfffffe00bb153f50
May 27 05:09:25 arch kernel: code segment               = base 0x0, limit
0xfffff, type 0x1b
May 27 05:09:25 arch kernel:                    = DPL 0, pres 1, long 1, def32
0, gran 1
May 27 05:09:25 arch kernel: processor eflags   = interrupt enabled, resume,
IOPL = 0
May 27 05:09:25 arch kernel: current process            = 97681 (nginx)
May 27 05:09:25 arch kernel: trap number                = 12
May 27 05:09:25 arch kernel: panic: page fault
May 27 05:09:25 arch kernel: cpuid = 0
May 27 05:09:25 arch kernel: time = 1653615832
May 27 05:09:25 arch kernel: KDB: stack backtrace:
May 27 05:09:25 arch kernel: #0 0xffffffff80c57535 at kdb_backtrace+0x65
May 27 05:09:25 arch kernel: #1 0xffffffff80c09f11 at vpanic+0x181
May 27 05:09:25 arch kernel: #2 0xffffffff80c09d83 at panic+0x43
May 27 05:09:25 arch kernel: #3 0xffffffff8108b1a7 at trap_fatal+0x387
May 27 05:09:25 arch kernel: #4 0xffffffff8108b1ff at trap_pfault+0x4f
May 27 05:09:25 arch kernel: #5 0xffffffff8108a85d at trap+0x27d
May 27 05:09:25 arch kernel: #6 0xffffffff81061f08 at calltrap+0x8
May 27 05:09:25 arch kernel: #7 0xffffffff80c9c38f at m_pullup+0x1af
May 27 05:09:25 arch kernel: #8 0xffffffff8211f2bf at ipfw_chk+0x3fcf
May 27 05:09:25 arch kernel: #9 0xffffffff8212105c at ipfw_check_frame+0x13c
May 27 05:09:25 arch kernel: #10 0xffffffff80d422c7 at pfil_run_hooks+0x97
May 27 05:09:25 arch kernel: #11 0xffffffff80d23bf4 at ether_output_frame+0x94
May 27 05:09:25 arch kernel: #12 0xffffffff80d23b08 at ether_output+0x6b8
May 27 05:09:25 arch kernel: #13 0xffffffff80db3ca5 at ip_output_send+0x75
May 27 05:09:25 arch kernel: #14 0xffffffff80db3ac2 at ip_output+0x12b2
May 27 05:09:25 arch kernel: #15 0xffffffff80dc9ab4 at tcp_output+0x1b04
May 27 05:09:25 arch kernel: #16 0xffffffff80ddb189 at tcp_usr_send+0x229
May 27 05:09:25 arch kernel: #17 0xffffffff80c07e2a at vn_sendfile+0x197a
May 27 05:09:25 arch kernel: Uptime: 56m29s

-- 
You are receiving this mail because:
You are the assignee for the bug.