git: 1d44514fcd68 - main - rmlock: Add a required compiler membar to the rlock slow path

[Mark Johnston]

The branch main has been updated by markj:

URL: https://cgit.FreeBSD.org/src/commit/?id=1d44514fcd68809cfd493a7352ace29ddad443d6

commit 1d44514fcd68809cfd493a7352ace29ddad443d6
Author:     Mark Johnston <markj at FreeBSD.org>
AuthorDate: 2021-02-24 02:15:50 +0000
Commit:     Mark Johnston <markj at FreeBSD.org>
CommitDate: 2021-02-24 02:17:12 +0000

    rmlock: Add a required compiler membar to the rlock slow path
    
    The tracker flags need to be loaded only after the tracker is removed
    from its per-CPU queue.  Otherwise, readers may fail to synchronize with
    pending writers attempting to propagate priority to active readers, and
    readers and writers deadlock on each other.  This was observed in a
    stable/12-based armv7 kernel where the compiler had reordered the load
    of rmp_flags to before the stores updating the queue.
    
    Reviewed by:    rlibby, scottl
    Discussed with: kib
    Sponsored by:   Rubicon Communications, LLC ("Netgate")
    MFC after:      1 week
    Differential Revision:  https://reviews.freebsd.org/D28821
---
 sys/kern/kern_rmlock.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/sys/kern/kern_rmlock.c b/sys/kern/kern_rmlock.c
index 401445d3d03f..9135709d88cf 100644
--- a/sys/kern/kern_rmlock.c
+++ b/sys/kern/kern_rmlock.c
@@ -362,7 +362,11 @@ _rm_rlock_hard(struct rmlock *rm, struct rm_priotracker *tracker, int trylock)
 	/* Remove our tracker from the per-cpu list. */
 	rm_tracker_remove(pc, tracker);
 
-	/* Check to see if the IPI granted us the lock after all. */
+	/*
+	 * Check to see if the IPI granted us the lock after all.  The load of
+	 * rmp_flags must happen after the tracker is removed from the list.
+	 */
+	__compiler_membar();
 	if (tracker->rmp_flags) {
 		/* Just add back tracker - we hold the lock. */
 		rm_tracker_add(pc, tracker);