svn commit: r352410 - head/sys/vm

[Mark Johnston]

Author: markj
Date: Mon Sep 16 15:12:49 2019
New Revision: 352410
URL: https://svnweb.freebsd.org/changeset/base/352410

Log:
  Fix a race in vm_page_dequeue_deferred_free() after r352110.
  
  This function loaded the page's queue index before setting PGA_DEQUEUE.
  In this window the page daemon may have deactivated the page, updating
  its queue index.  Make the operation atomic using vm_page_pqstate_cmpset();
  the page daemon will not modify the page once it observes that PGA_DEQUEUE
  is set.
  
  Reported and tested by:	pho
  Reviewed by:	alc, kib
  Sponsored by:	Netflix
  Differential Revision:	https://reviews.freebsd.org/D21639

Modified:
  head/sys/vm/vm_page.c
  head/sys/vm/vm_page.h

Modified: head/sys/vm/vm_page.c
==============================================================================
--- head/sys/vm/vm_page.c	Mon Sep 16 15:09:31 2019	(r352409)
+++ head/sys/vm/vm_page.c	Mon Sep 16 15:12:49 2019	(r352410)
@@ -3315,13 +3315,18 @@ vm_page_dequeue_deferred_free(vm_page_t m)
 
 	KASSERT(m->ref_count == 0, ("page %p has references", m));
 
-	if ((m->aflags & PGA_DEQUEUE) != 0)
-		return;
-	atomic_thread_fence_acq();
-	if ((queue = m->queue) == PQ_NONE)
-		return;
-	vm_page_aflag_set(m, PGA_DEQUEUE);
-	vm_page_pqbatch_submit(m, queue);
+	for (;;) {
+		if ((m->aflags & PGA_DEQUEUE) != 0)
+			return;
+		atomic_thread_fence_acq();
+		if ((queue = atomic_load_8(&m->queue)) == PQ_NONE)
+			return;
+		if (vm_page_pqstate_cmpset(m, queue, queue, PGA_DEQUEUE,
+		    PGA_DEQUEUE)) {
+			vm_page_pqbatch_submit(m, queue);
+			break;
+		}
+	}
 }
 
 /*

Modified: head/sys/vm/vm_page.h
==============================================================================
--- head/sys/vm/vm_page.h	Mon Sep 16 15:09:31 2019	(r352409)
+++ head/sys/vm/vm_page.h	Mon Sep 16 15:12:49 2019	(r352410)
@@ -783,8 +783,6 @@ vm_page_pqstate_cmpset(vm_page_t m, uint32_t oldq, uin
 {
 	uint32_t *addr, nval, oval, qsmask;
 
-	vm_page_assert_locked(m);
-
 	fflags <<= VM_PAGE_AFLAG_SHIFT;
 	nflags <<= VM_PAGE_AFLAG_SHIFT;
 	newq <<= VM_PAGE_QUEUE_SHIFT;