svn commit: r367849 - head/sys/kern

[Mark Johnston]

Author: markj
Date: Thu Nov 19 18:37:28 2020
New Revision: 367849
URL: https://svnweb.freebsd.org/changeset/base/367849

Log:
  callout(9): Fix a race between CPU migration and callout_drain()
  
  Suppose a running callout re-arms itself, and before the callout
  finishes running another CPU calls callout_drain() and goes to sleep.
  softclock_call_cc() will wake up the draining thread, which may not run
  immediately if there is a lot of CPU load.  Furthermore, the callout is
  still in the callout wheel so it can continue to run and re-arm itself.
  Then, suppose that the callout migrates to another CPU before the
  draining thread gets a chance to run.  The draining thread is in this
  loop in _callout_stop_safe():
  
  	while (cc_exec_curr(cc) == c) {
  		CC_UNLOCK(cc);
  		sleep();
  		CC_LOCK(cc);
  	}
  
  but after the migration, cc points to the wrong CPU's callout state.
  Then the draining thread goes off and removes the callout from the
  wheel, but does so using the wrong lock and per-CPU callout state.
  
  Fix the problem by doing a re-lookup of the callout CPU after sleeping.
  
  Reported by:	syzbot+79569cd4d76636b2cc1c at syzkaller.appspotmail.com
  Reported by:	syzbot+1b27e0237aa22d8adffa at syzkaller.appspotmail.com
  Reported by:	syzbot+e21aa5b85a9aff90ef3e at syzkaller.appspotmail.com
  Reviewed by:	emaste, hselasky
  Tested by:	pho
  MFC after:	1 week
  Sponsored by:	The FreeBSD Foundation
  Differential Revision:	https://reviews.freebsd.org/D27266

Modified:
  head/sys/kern/kern_timeout.c

Modified: head/sys/kern/kern_timeout.c
==============================================================================
--- head/sys/kern/kern_timeout.c	Thu Nov 19 18:03:40 2020	(r367848)
+++ head/sys/kern/kern_timeout.c	Thu Nov 19 18:37:28 2020	(r367849)
@@ -1145,7 +1145,7 @@ again:
 			 * just wait for the current invocation to
 			 * finish.
 			 */
-			while (cc_exec_curr(cc, direct) == c) {
+			if (cc_exec_curr(cc, direct) == c) {
 				/*
 				 * Use direct calls to sleepqueue interface
 				 * instead of cv/msleep in order to avoid
@@ -1193,7 +1193,7 @@ again:
 
 				/* Reacquire locks previously released. */
 				PICKUP_GIANT();
-				CC_LOCK(cc);
+				goto again;
 			}
 			c->c_flags &= ~CALLOUT_ACTIVE;
 		} else if (use_lock &&