help w/panic under heavy load - 5.4

[Giorgos Keramidas]

On 2005-07-21 14:57, Giorgos Keramidas <keramida at freebsd.org> wrote:
> On 2005-07-20 11:41, Edwin <edwin at verolan.com> wrote:
> > I'm trying to understand the particulars about this - I get the null pointer
> > part, but as to ip_fragment - it's fragmenting mbufs to handle ip packets
> > during switching? and its failing trying to copy data past the end of the
> > chain?
>
> ip_fastfwd() thinks that it should fragment the packet because it somehow
> calculates a bogus ``mtu'' value.  See the mtu value in frame 12 of the stack
> trace below.
>
> > #10 0xc0611fef in panic (fmt=0xc0820008 "default") at /usr/src/sys/kern/kern_shutdown.c:550
> > #11 0xc0641a2c in m_copym (m=0x0, off0=1500, len=1480, wait=1)
> >     at /usr/src/sys/kern/uipc_mbuf.c:385
> > #12 0xc069b694 in ip_fragment (ip=0xc11bd80e, m_frag=0xc76bfc6c, mtu=-1056787456,
> >     if_hwassist_flags=0, sw_csum=1) at /usr/src/sys/netinet/ip_output.c:967
>
> The ``mtu'' is an extremely small integer value, which is definitely a problem
> here.  Somehow, ip_fastforward() calculates a very wrong value for the ``mtu''.

The check for finding the right MTU in ip_output.c is a bit different,
as it includes a check for RTF_UP:

 777:        if (ip->ip_off & IP_DF) {
 778:                error = EMSGSIZE;
 779:                /*
 780:                 * This case can happen if the user changed the MTU
 781:                 * of an interface after enabling IP on it.  Because
 782:                 * most netifs don't keep track of routes pointing to
 783:                 * them, there is no way for one to update all its
 784:                 * routes when the MTU is changed.
 785:                 */
 786:                if ((ro->ro_rt->rt_flags & (RTF_UP | RTF_HOST)) &&
 787:                    (ro->ro_rt->rt_rmx.rmx_mtu > ifp->if_mtu)) {
 788:                        ro->ro_rt->rt_rmx.rmx_mtu = ifp->if_mtu;
 789:                }
 790:                ipstat.ips_cantfrag++;
 791:                goto bad;
 792:        }

The check for RTF_UP doesn't exist in ip_fastfwd.c, except perhaps
through the ip_findroute() call.  I'm probably confused, but it seems
that ip_findroute() in ip_fastfwd.c may still return a route entry for a
gateway that is not yet RTF_UP, since the check for the RTF_UP flag is
not done for the dst.rt_gateway route entry too.

This may be the cause of the invalid MTU value you're seeing.  Can you
try the following patch for ip_fastfwd.c?

The diff is also available online at:
http://people.freebsd.org/~keramida/diff/fastfwd-mtu.patch

%%%
Index: ip_fastfwd.c
===================================================================
RCS file: /home/ncvs/src/sys/netinet/ip_fastfwd.c,v
retrieving revision 1.28
diff -u -r1.28 ip_fastfwd.c
--- ip_fastfwd.c	4 May 2005 13:09:19 -0000	1.28
+++ ip_fastfwd.c	21 Jul 2005 14:38:35 -0000
@@ -537,12 +537,13 @@
 	}
 
 	/*
-	 * Check if packet fits MTU or if hardware will fragement for us
+	 * Check if packet fits MTU or if hardware will fragment for us.
+	 * If necessary, update the MTU of the route entry too.
 	 */
-	if (ro.ro_rt->rt_rmx.rmx_mtu)
-		mtu = min(ro.ro_rt->rt_rmx.rmx_mtu, ifp->if_mtu);
-	else
-		mtu = ifp->if_mtu;
+	if ((ro.ro_rt.rt_flags & (RTF_UP | RTF_HOST)) &&
+	    ro.ro_rt->rt_rmx.rmx_mtu > ipf->if_mtu)
+		ro.ro_rt->rt_rmx.rmx_mtu = ipf->if_mtu;
+	mtu = ifp->if_mtu;
 
 	if (ip->ip_len <= mtu ||
 	    (ifp->if_hwassist & CSUM_FRAGMENT && (ip->ip_off & IP_DF) == 0)) {
%%%