panic: rtqkill route really not free on freebsd 8.0-release update

[Xin LI]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi, Bjoern,

On 2010/07/02 01:39, Bjoern A. Zeeb wrote:
> On Sat, 5 Jun 2010, Chao Shin wrote:
> 
> Hey,
> 
>> We add kdb/ddb and extra panic info printing into kernel and catch
>> this panic again.
>>
>> We have instrumented the kernel and found that this panic happens when
>> draining == 1,
>> but seems to be confused with the fact that all access to radix trees
>> are protected
>> by locks.  Can anyone familiar with these code shed us some light on
>> this?
>>
>> below is url to screenshot in ddb:
>> http://www.delphij.net/zhao/1.png
>> http://www.delphij.net/zhao/2.png
> 
> Did anyone pick this up?

I don't think so.

Currently we believe that there is some call paths that would exhibit
the following:

Thread A		Thread B
			(...)
RTLOCK(rt)
rt->ref--;
[ref drops to 0 now]
			(obtain rnh_lock)
			(in in_matroute)
			saw rt->ref == 0
			rt->rt_flags & RTPRF_OURS == 0
			(return from in_matroute())
			RT_LOCK(rt) <-- blocks here
rt->rt_flags |= OURS
RT_UNLOCK(rt);
			RT_LOCK(rt) <-- got a wakeup
			rt->ref++
			(ref == 1 && rt->rt_flags & RTPRF_OURS)

With the attached workaround they have not see this type of panics so
far but that doesn't seem ideal.

Kip and Qing's paper titled "Optimizing the BSD routing system for
parallel processing" suggests copying the route entry rather than
referencing it but I didn't yet on how should I implement that and do
benchmark...

Cheers,
- -- 
Xin LI <delphij at delphij.net>	http://www.delphij.net/
FreeBSD - The Power to Serve!	       Live free or die
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.15 (FreeBSD)

iQEcBAEBCAAGBQJMLlmNAAoJEATO+BI/yjfBzvAIANjmEXX54lryJ6Qq37yUFdmd
BQqw7r/Q7IYD6gOBU0/iMUySa4x6H3U+8TPUK8Rf+ARkG8CP3JsRMPJtLkFs5Eby
lmvQDRcfcKzFCAC40m/FmdlCl0c2Q/mz5H4PYve3zuU+BEDN0NOEIUtnYVmOJK1U
4O5XXZcAzNT1BXKKwbogwQq0t4dhT/3+4inH6vC3w8HpzwDfXS2GogFSOYlSurvC
h7b2wjrD7sgTPZZj1DN7qWjGSRNAao+AGzlzvQR6tNCqWV+bn8qF+QaNoFepev+g
ITeUh9IXffn646WCRF5whKUjz+M9IvSPhqGiFyWfhcGj8DbDt074XMsHiBLh7nc=
=lHSK
-----END PGP SIGNATURE-----
-------------- next part --------------
Index: in_rmx.c
===================================================================
--- in_rmx.c	(revision 208681)
+++ in_rmx.c	(working copy)
@@ -121,12 +121,12 @@
 	struct radix_node *rn = rn_match(v_arg, head);
 	struct rtentry *rt = (struct rtentry *)rn;
 
-	/*XXX locking? */
-	if (rt && rt->rt_refcnt == 0) {		/* this is first reference */
-		if (rt->rt_flags & RTPRF_OURS) {
-			rt->rt_flags &= ~RTPRF_OURS;
-			rt->rt_rmx.rmx_expire = 0;
-		}
+	if (rt && rt->rt_refcnt == 0 &&	/* this is first reference */
+	    rt->rt_flags & RTPRF_OURS) {
+		RT_LOCK(rt);
+		rt->rt_flags &= ~RTPRF_OURS;
+		rt->rt_rmx.rmx_expire = 0;
+		RT_UNLOCK(rt);
 	}
 	return rn;
 }
@@ -206,6 +206,7 @@
 
 	RADIX_NODE_HEAD_WLOCK_ASSERT(ap->rnh);
 
+	RT_LOCK(rt);
 	if (rt->rt_flags & RTPRF_OURS) {
 		ap->found++;
 
@@ -234,6 +235,7 @@
 					    rt->rt_rmx.rmx_expire);
 		}
 	}
+	RT_UNLOCK(rt);
 
 	return 0;
 }