Enabling NUMA in BIOS stop booting FreeBSD

[Konstantin Belousov]

On Tue, Dec 13, 2016 at 03:49:32PM +0300, Slawa Olhovchenkov wrote:
> > Boot with NUMA enabled and interleave off.
> 
> Already with patched kernel
> 
> > Patch kernel with the 'if (1 || ...)' patch.
> > Reboot, enter BIOS setup and enable interleave there.
> > Try to boot - does it boot ?
> 
> No.
> 
> > If it did not booted, power machine off for 10 minutes.
> 
> OK
> 
> > Power it on, try to boot (with the same patched kernel).
> > Does the machine boot now ?
> 
> Don't boot.

I am really puzzled.  In other words, touching all memory causes the
msgbuf to not hang.

Can you try one more experiment ?
Take the patch below, apply it.
>From the config where interleave is disabled, install new kernel.
Reboot, enter BIOS setup and enable interleave.
Set late_console to zero in loader.
Do not enable memory test.
Boot the patched kernel.
Kernel must hang, according to your previous reports.
I want to see the console log.

diff --git a/sys/kern/subr_msgbuf.c b/sys/kern/subr_msgbuf.c
index f275aef3b4f..1be7a629f65 100644
--- a/sys/kern/subr_msgbuf.c
+++ b/sys/kern/subr_msgbuf.c
@@ -67,14 +67,19 @@ msgbuf_init(struct msgbuf *mbp, void *ptr, int size)
 	mbp->msg_ptr = ptr;
 	mbp->msg_size = size;
 	mbp->msg_seqmod = SEQMOD(size);
+printf("YYY1\n");
 	msgbuf_clear(mbp);
+printf("YYY2\n");
 	mbp->msg_magic = MSG_MAGIC;
 	mbp->msg_lastpri = -1;
 	mbp->msg_flags = 0;
+printf("YYY3\n");
 	bzero(&mbp->msg_lock, sizeof(mbp->msg_lock));
 	mtx_init(&mbp->msg_lock, "msgbuf", NULL, MTX_SPIN);
+printf("YYY4\n");
 }
 
+
 /*
  * Reinitialize a message buffer, retaining its previous contents if
  * the size and checksum are correct. If the old contents cannot be
@@ -85,8 +90,10 @@ msgbuf_reinit(struct msgbuf *mbp, void *ptr, int size)
 {
 	u_int cksum;
 
-	if (mbp->msg_magic != MSG_MAGIC || mbp->msg_size != size) {
+	if (1 || mbp->msg_magic != MSG_MAGIC || mbp->msg_size != size) {
+printf("XXX1\n");
 		msgbuf_init(mbp, ptr, size);
+printf("XXX2\n");
 		return;
 	}
 	mbp->msg_seqmod = SEQMOD(size);
@@ -117,10 +124,12 @@ void
 msgbuf_clear(struct msgbuf *mbp)
 {
 
+printf("ZZZ1\n");
 	bzero(mbp->msg_ptr, mbp->msg_size);
 	mbp->msg_wseq = 0;
 	mbp->msg_rseq = 0;
 	mbp->msg_cksum = 0;
+printf("ZZZ2\n");
 }
 
 /*