Panic in the udp_input() under heavy load

[Maxim Sobolev]

Hi Gang,

We are seeing repeatable panics under high PPS load on our production 
systems. It happens when the traffic gets into the range or 200MBps and 
150-200K PPS. We have been managed to track it down to the following 
piece of code:

(gdb) l *udp_input+0x5d2
0xffffffff806f6202 is in udp_input (/usr/src/sys/netinet/udp_usrreq.c:628).
623             if (inp->inp_ip_minttl && inp->inp_ip_minttl > ip->ip_ttl) {
624                     INP_RUNLOCK(inp);
625                     goto badunlocked;
626             }
627             up = intoudpcb(inp);
628             if (up->u_tun_func == NULL) {
629                     udp_append(inp, ip, m, iphlen + sizeof(struct 
udphdr), &udp_in);
630             } else {
631                     /*
632                      * Engage the tunneling protocol.

The faulty line appears to be 628, with up value is being NULL, attempt 
to deference it causes NULL pointer exception. I believe this particular 
piece of code has been introduced here:

---
Author: bz
Date: Thu Aug 13 15:16:30 2009
New Revision: 196192
URL: http://svn.freebsd.org/changeset/base/196192

Log:
   MFC: r192649

     Implement UDP control block support.

     Add udpcb support with own fields and flags for UDP instead
     of further sticking things into in_pcb and flags fields.
     Attach the udpcb to the inp_ppcb in the kernel.

     Note: the udp tunneling parts are not (yet) existing in 7
     and thus were not merged.

   Reviewed by:	rwatson
---

The screenshot of the panic message is attached. This is pretty recent 
8.2-STABLE. Any help is greatly appreciated. This particular bug has 
haunted us for at least 4-5 months now.

Thanks!

-Maxim