i386/119356: i386 ACPI wakeup not work due resource exhaustion

[Dan Lukes]

>Number:         119356
>Category:       i386
>Synopsis:       i386 ACPI wakeup not work due resource exhaustion
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-i386
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sat Jan 05 11:50:01 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator:     Dan Lukes
>Release:        FreeBSD 6.3-PRERELEASE i386
>Organization:
Obludarium
>Environment:
System: FreeBSD 6.3-PRERELEASE
but it seems to apply to RELENG_7 and HEAD as well

>Description:
	acpi module register acpi_alloc_wakeup_handler() routine to be run
during startup.

	The routine shall alloc one page of physical memory in the 
address range of 0-9FFFFh (to be avaiable in CPU's real mode)

	Unfortunately, the routine is called to late during initialization
sequence, so all potential pages may be allocated already

	The routine is registered with SI_SUB_KMEM/SI_ORDER_ANY priority, 
but it's part of loadable modules. All modules initialization routines 
are deferred until preload_linker(), so the effective priority of
acpi_alloc_wakeup_handler() is worse than SI_SUB_KLD despite of declared priority.

	On my system, there are 158 pages in the range 0-9FFFFh. After VM 
initialization 155 of them are free. 
kmeminit() eat 70 of them, 38 pages disapear during several invocations of
malloc_init(), the scmeminit() grab latest 47 pages. 

	After scmeminit() all pages in the range 0-9FFFh are on PQ_NONE
queue and are not avaiable for further allocation.

	It's long time before the acpi's acpi_alloc_wakeup_handler() invoked.

	At the time of invocation, no required pages are avaiable so
acpi_alloc_wakeup_handler() fail. As a result, no machinde-dependent wake-up
routine can be installed. It broke the system wakeup.

	The problem occur on i386 architecture only.

>How-To-Repeat:
>Fix:

	The best fix is to rewrite system initialization scheduler within
mi_startup() 

	The routines shall be invoked according declared priority despite
they are compiled-in or part of preloaded module.

	Unfortunately it require reengineering of preload_linker()
and other linker related code.

	I can't do it.

	Worse, but still good, we can rewrite the physical memory management
to save the 0-9FFFFh pages as long as possible. Currently they are exhausted
despite other high-address pages are still avaiable.

	Unfortunately I have insuficient knowledge of memory allocator, so
I can't supply patch this way.

	I can supply the hack. Yes, it is hack, but IMHO acceptable hack. The
acpi_alloc_wakeup_handler() needs to be moved out of module into statically
compiled code. The routine in the compiled-in code will be invoked according 
declared priority. 

	Routine on it's original place initialise three static variables
acpi_waketag/acpi_wakemap/acpi_wakeaddr which is used by other ACPI's routines.

	Moved routine declare the same variables as global, so they are
avaiable to acpi module as well. Variables declared inside of ACPI module 
needs to be changed to external.

	No other modification needed.

	The hack has disadvantage. At the time of moved
acpi_alloc_wakeup_handler() invocation the system doesn't know the ACPI module
is preloaded or not. So page may be allocated even acpi not present.

	We can try to determine the ACPI will be loaded later or we can create
the hook invoked after the module initialisation which free the page if acpi
not present. 

	I give up on it. It's hack. The one wasted page is the price for it.

	Without a fix the wake won't work correctly.

	I decide to move the routine into sys/i386/i386/vm_machdep.c

	There are may be better place.

--- sys/i386/acpica/acpi_wakeup.c.ORIG	2008-01-05 11:04:04.000000000 +0100
+++ sys/i386/acpica/acpi_wakeup.c	2008-01-05 11:12:10.000000000 +0100
@@ -294,40 +294,9 @@
 	return (ret);
 }
 
-static bus_dma_tag_t	acpi_waketag;
-static bus_dmamap_t	acpi_wakemap;
-static vm_offset_t	acpi_wakeaddr;
-
-static void
-acpi_alloc_wakeup_handler(void)
-{
-	void *wakeaddr;
-
-	if (!cold)
-		return;
-
-	/*
-	 * Specify the region for our wakeup code.  We want it in the low 1 MB
-	 * region, excluding video memory and above (0xa0000).  We ask for
-	 * it to be page-aligned, just to be safe.
-	 */
-	if (bus_dma_tag_create(/*parent*/ NULL,
-	    /*alignment*/ PAGE_SIZE, /*no boundary*/ 0,
-	    /*lowaddr*/ 0x9ffff, /*highaddr*/ BUS_SPACE_MAXADDR, NULL, NULL,
-	    /*maxsize*/ PAGE_SIZE, /*segments*/ 1, /*maxsegsize*/ PAGE_SIZE,
-	    0, busdma_lock_mutex, &Giant, &acpi_waketag) != 0) {
-		printf("acpi_alloc_wakeup_handler: can't create wake tag\n");
-		return;
-	}
-	if (bus_dmamem_alloc(acpi_waketag, &wakeaddr, BUS_DMA_NOWAIT,
-	    &acpi_wakemap) != 0) {
-		printf("acpi_alloc_wakeup_handler: can't alloc wake memory\n");
-		return;
-	}
-	acpi_wakeaddr = (vm_offset_t)wakeaddr;
-}
-
-SYSINIT(acpiwakeup, SI_SUB_KMEM, SI_ORDER_ANY, acpi_alloc_wakeup_handler, 0)
+extern bus_dma_tag_t	acpi_waketag;
+extern bus_dmamap_t	acpi_wakemap;
+extern vm_offset_t	acpi_wakeaddr;
 
 static void
 acpi_realmodeinst(void *arg, bus_dma_segment_t *segs, int nsegs, int error)
--- sys/i386/i386/vm_machdep.c.ORIG	2008-01-05 11:04:16.000000000 +0100
+++ sys/i386/i386/vm_machdep.c	2008-01-05 11:09:18.000000000 +0100
@@ -69,7 +69,11 @@
 #include <sys/unistd.h>
 #include <sys/vnode.h>
 #include <sys/vmmeter.h>
+#include <sys/bus.h>
 
+#include <vm/vm.h>
+
+#include <machine/bus.h>
 #include <machine/cpu.h>
 #include <machine/cputypes.h>
 #include <machine/md_var.h>
@@ -849,3 +853,39 @@
 
 	return 1;
 }
+
+bus_dma_tag_t	acpi_waketag;
+bus_dmamap_t	acpi_wakemap;
+vm_offset_t	acpi_wakeaddr = NULL;
+
+static void
+acpi_alloc_wakeup_handler(void)
+{
+	void *wakeaddr;
+
+	if (!cold)
+		return;
+
+	/*
+	 * Specify the region for our wakeup code.  We want it in the low 1 MB
+	 * region, excluding video memory and above (0xa0000).  We ask for
+	 * it to be page-aligned, just to be safe.
+	 */
+	if (bus_dma_tag_create(/*parent*/ NULL,
+	    /*alignment*/ PAGE_SIZE, /*no boundary*/ 0,
+	    /*lowaddr*/ 0x9ffff, /*highaddr*/ BUS_SPACE_MAXADDR, NULL, NULL,
+	    /*maxsize*/ PAGE_SIZE, /*segments*/ 1, /*maxsegsize*/ PAGE_SIZE,
+	    0, busdma_lock_mutex, &Giant, &acpi_waketag) != 0) {
+		printf("acpi_alloc_wakeup_handler: can't create wake tag\n");
+		return;
+	}
+	if (bus_dmamem_alloc(acpi_waketag, &wakeaddr, BUS_DMA_NOWAIT,
+	    &acpi_wakemap) != 0) {
+		printf("acpi_alloc_wakeup_handler: can't alloc wake memory\n");
+		return;
+	}
+	acpi_wakeaddr = (vm_offset_t)wakeaddr;
+}
+
+SYSINIT(acpiwakeup, SI_SUB_KMEM, SI_ORDER_ANY, acpi_alloc_wakeup_handler, 0)
+
>Release-Note:
>Audit-Trail:
>Unformatted: