svn commit: r309071 - in stable: 10/sys/i386/i386 11/sys/i386/i386

[John Baldwin]

Author: jhb
Date: Wed Nov 23 23:45:42 2016
New Revision: 309071
URL: https://svnweb.freebsd.org/changeset/base/309071

Log:
  MFC 307975: Enable EFER_NXE properly on APs.
  
  EFER_NXE is set in the EFER MSR by initializecpu() and must be set on all
  CPUs in the system.  When PG_NX support was added to PAE on i386, the
  block to enable EFER_NXE was placed in a section of initializecpu() that
  only runs if 'cpu == CPU_686'.  During early boot, locore does an
  initial pass to set cpu that sets it to CPU_686 on all CPUs later than
  a Pentium.  Later, printcpuinfo() adjusts the 'cpu' variable on
  PII and later CPUs to one of CPU_PII, CPU_PIII, or CPU_P4.  However,
  printcpuinfo() is called after initializecpu() on the BSP, so the BSP
  would enable EFER_NXE and pg_nx.  The APs execute initializecpu() much
  later after printcpuinfo() has run.  The end result on a modern CPU was
  that cpu was set to CPU_PIII when the APs invoked initializecpu(), so
  they did not enable EFER_NXE.  As a result, the APs would fault when
  trying to access any pages marked with PG_NX set.
  
  When booting a 2 CPU PAE kernel in bhyve this manifested as a hang before
  single user mode.  The attempt to execute /bin/init tried to copy out
  the exec strings (argv, etc.) to a non-executable mapping while running
  on the AP.  The instruction kept faulting due to invalid bits in the PTE
  in an infinite loop.
  
  Fix this by moving the code to enable EFER_NXE out of the switch statement
  on 'cpu' and always doing it if 'amd_feature' supports AMDID_NX.

Modified:
  stable/11/sys/i386/i386/initcpu.c
Directory Properties:
  stable/11/   (props changed)

Changes in other areas also in this revision:
Modified:
  stable/10/sys/i386/i386/initcpu.c
Directory Properties:
  stable/10/   (props changed)

Modified: stable/11/sys/i386/i386/initcpu.c
==============================================================================
--- stable/11/sys/i386/i386/initcpu.c	Wed Nov 23 22:57:47 2016	(r309070)
+++ stable/11/sys/i386/i386/initcpu.c	Wed Nov 23 23:45:42 2016	(r309071)
@@ -753,16 +753,6 @@ initializecpu(void)
 			init_transmeta();
 			break;
 		}
-#if defined(PAE) || defined(PAE_TABLES)
-		if ((amd_feature & AMDID_NX) != 0) {
-			uint64_t msr;
-
-			msr = rdmsr(MSR_EFER) | EFER_NXE;
-			wrmsr(MSR_EFER, msr);
-			pg_nx = PG_NX;
-			elf32_nxstack = 1;
-		}
-#endif
 		break;
 #endif
 	default:
@@ -774,6 +764,16 @@ initializecpu(void)
 		cpu_fxsr = hw_instruction_sse = 1;
 	}
 #endif
+#if defined(PAE) || defined(PAE_TABLES)
+	if ((amd_feature & AMDID_NX) != 0) {
+		uint64_t msr;
+
+		msr = rdmsr(MSR_EFER) | EFER_NXE;
+		wrmsr(MSR_EFER, msr);
+		pg_nx = PG_NX;
+		elf32_nxstack = 1;
+	}
+#endif
 }
 
 void