kernel panic with netgraph and mpd5.8
Donald Baud
donaldbaud at yahoo.com
Tue Jul 12 10:28:33 UTC 2016
>> On 2016-07-10 10:49, Donald Baud via freebsd-net wrote:
>>> Hi I'm running an l2tp lns through mpd5.8 and it's been crashing
>>> twice in 24h.
>>> This is a new project replacing a cisco 7206, 700-sessions 800mbit/s
>>>
>>>
>>> I am not familiar with troubleshooting kernel panic's,
>>>
>>> I am suspecting that the crash is happening inside the netgraph
>>> module because the crash is happening at the
>>>
>>> instruction pointer = 0x20:0xffffffff81c38283
>>> I included the 2 two crash logs. I need some help to to figure out
>>> what to do next.
>>>
>>> -Dbaud
>>>
>>>
>> On 7/10/16 5:14 PM, Hooman Fazaeli wrote:
>> - Upgrade to mpd 5 (/usr/ports/net/mpd5)
>> - Try below workarounds:
>> https://lists.freebsd.org/pipermail/freebsd-bugs/2014-June/056548.html
>> https://lists.freebsd.org/pipermail/freebsd-bugs/2014-June/056549.html
>> https://lists.freebsd.org/pipermail/freebsd-net/2014-June/038954.html
>>
>>
> On 7/10/16 8:43 PM, Donald Baud via freebsd-net wrote:
> - I'm already using the latest mpd5:
> > mpd5 --version
> Version 5.8 (root at 101amd64-quarterly-job-15 12:36 5-Jun-2016)
>
> - I had already reviewed those links you mentioned.
> Here is a summary of the main suggestions in them.
> * Add a "sleep 1" to up-down interface events.
> * Revert to RELENG8 or 9
> * boost net.graph sysctl/loader.conf
> net.graph.maxdata=262140 # /boot/loader.conf
> net.graph.maxalloc=262140 # /boot.loader.conf
>
> I was using the following tunings
> net.graph.maxdgram=524288 (via sysctl.conf default=20480)
> net.graph.recvspace=524288 (via sysctl.conf default=20480)
> net.graph.maxdata=65536 (via loader.conf default=4096 )
> net.graph.maxalloc=65536 (via loader.conf default=4096 )
>
> I am suspecting that the panic might be caused by a too high maxdata
> and maxalloc values:
> I reduced the value to 20480, I'll report back if that will reduce the
> occurence of kernel panics.
>
>> vmstat -z | head -1 ; vmstat -z | grep -i graph
> ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP
> NetGraph items: 72, 20491, 2, 1672,467166841, 0, 0
> NetGraph data items: 72, 20491, 0, 1643,1240166475, 0, 0
>
>
The server crashed again this morning.
It looks like it crashes somewhere in the netgraph.ko module
Could someone please help me troubleshoot this issue, it crashes around
the same location
instruction pointer = 0x20:0xffffffff81c3828d
The crash happens at random times not necessarily under heavy load.
- using plain GENERIC kernel
10.3-RELEASE-p4 FreeBSD 10.3-RELEASE-p4 #0: Sat May 28 12:23:44 UTC 2016
root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
- # kldstat
Id Refs Address Size Name
1 32 0xffffffff80200000 17bc6a8 kernel
2 2 0xffffffff81c11000 114db ipfw.ko
3 1 0xffffffff81c23000 d32f dummynet.ko
4 1 0xffffffff81c31000 3831 ng_socket.ko
5 8 0xffffffff81c35000 ba02 netgraph.ko
6 1 0xffffffff81c41000 2b99 ng_mppc.ko
7 1 0xffffffff81c44000 80c rc4.ko
8 1 0xffffffff81c45000 23dc vmmemctl.ko
9 1 0xffffffff81c48000 397d ng_l2tp.ko
10 1 0xffffffff81c4c000 4b04 ng_ksocket.ko
11 1 0xffffffff81c51000 17d6 ng_tee.ko
12 1 0xffffffff81c53000 40d2 ng_iface.ko
13 1 0xffffffff81c58000 5829 ng_ppp.ko
14 1 0xffffffff81c5e000 18b1 ng_tcpmss.ko
- /etc/rc.conf
mpd_enable="YES"
quagga_daemons="zebra ospfd"
devd_enable="NO"
ipv6_network_interfaces="none"
ip6addrctl_enable="NO"
- /etc/sysctl.conf
net.inet.ip.fastforwarding=1
hw.intr_storm_threshold=40000
net.graph.maxdgram=524288
net.graph.recvspace=524288
- /boot/loader.conf
net.graph.maxdata=20480
net.graph.maxalloc=20480
- grep kernel: /var/log/messages
Jul 12 04:18:05 mybox syslogd: kernel boot file is /boot/kernel/kernel
Jul 12 04:18:05 mybox kernel:
Jul 12 04:18:05 mybox kernel:
Jul 12 04:18:05 mybox kernel: Fatal trap 9: general protection fault
while in kernel mode
Jul 12 04:18:05 mybox kernel: cpuid = 0; apic id = 00
Jul 12 04:18:05 mybox kernel: instruction pointer =
0x20:0xffffffff81c3828d
Jul 12 04:18:05 mybox kernel: stack pointer =
0x28:0xfffffe0174da8380
Jul 12 04:18:05 mybox kernel: frame pointer =
0x28:0xfffffe0174da83c0
Jul 12 04:18:05 mybox kernel: code segment = base 0x0, limit
0xfffff, type 0x1b
Jul 12 04:18:05 mybox kernel: = DPL 0, pres 1, long 1, def32 0, gran 1
Jul 12 04:18:05 mybox kernel: processor eflags = interrupt enabled,
resume, IOPL = 0
Jul 12 04:18:05 mybox kernel: current process = 659 (ng_queue3)
Jul 12 04:18:05 mybox kernel: trap number = 9
Jul 12 04:18:05 mybox kernel: panic: general protection fault
Jul 12 04:18:05 mybox kernel: cpuid = 0
Jul 12 04:18:05 mybox kernel: KDB: stack backtrace:
Jul 12 04:18:05 mybox kernel: #0 0xffffffff8098e390 at kdb_backtrace+0x60
Jul 12 04:18:05 mybox kernel: #1 0xffffffff80951066 at vpanic+0x126
Jul 12 04:18:05 mybox kernel: #2 0xffffffff80950f33 at panic+0x43
Jul 12 04:18:05 mybox kernel: #3 0xffffffff80d55f7b at trap_fatal+0x36b
Jul 12 04:18:05 mybox kernel: #4 0xffffffff80d55bfd at trap+0x77d
Jul 12 04:18:05 mybox kernel: #5 0xffffffff80d3b8d2 at calltrap+0x8
Jul 12 04:18:05 mybox kernel: #6 0xffffffff81c49606 at
ng_l2tp_rcvdata_lower+0x946
Jul 12 04:18:05 mybox kernel: #7 0xffffffff81c370ca at ng_apply_item+0x21a
Jul 12 04:18:05 mybox kernel: #8 0xffffffff81c36d1a at ng_snd_item+0x38a
Jul 12 04:18:05 mybox kernel: #9 0xffffffff81c4d3e2 at
ng_ksocket_incoming2+0x2f2
Jul 12 04:18:05 mybox kernel: #10 0xffffffff81c36f62 at ng_apply_item+0xb2
Jul 12 04:18:05 mybox kernel: #11 0xffffffff81c38d39 at ngthread+0x1b9
Jul 12 04:18:05 mybox kernel: #12 0xffffffff8091a4ea at fork_exit+0x9a
Jul 12 04:18:05 mybox kernel: #13 0xffffffff80d3be0e at fork_trampoline+0xe
Jul 12 04:18:05 mybox kernel: Uptime: 1d19h53m59s
Jul 12 04:18:05 mybox kernel: Dumping 560 out of 6119
MB:..3%..12%..23%..32%..43%..52%..63%..72%..83%..92%
Jul 12 04:18:05 mybox kernel: Dump complete
Jul 12 04:18:05 mybox kernel: [...]
More information about the freebsd-net
mailing list