Semi-working patch for amd64 suspend/resume

[Jung-uk Kim]

On Sunday 30 November 2008 03:10 pm, Alexander Motin wrote:
> Hi.
>
> Alexander Motin wrote:
> > Jung-uk Kim wrote:
> >> I was working on suspend/resume support for amd64 and this is
> >> the result.  It works with a modified QEMU (QEMU does not
> >> support S3) but real boxes that I have don't seem to like it
> >> (e.g., broken BIOSes). If there is someone interested in
> >> finishing it off or giving it a try, the patch is here:
> >>
> >> http://people.freebsd.org/~jkim/amd64_suspend.diff
> >
> > I have tried it on my Acer TM6292. S1/S2 are unsupported. On S3
> > system successfully got down, but on wakeup button, two seconds
> > after power up, even without video initialization, it shut down,
> > reset and then started usual boot. I have tried both original and
> > updated BIOS, without any difference.
> >
> > Can I give you any other help?
>
> I have spent a day investigating the problem. I was inserting empty
> infinite loop into the different points of wakeup process trying to
> find the place where system reboots. I just haven't found any other
> feedback channel as video is not initialized and beeper is not
> working for some reason.
>
> As result, I have found, that if I am inserting:
> qqq:
>
>          jmp     qqq
> lines before line 98 of acpi_switch.S:
> 	movl    $MSR_MTRRdefType, %ecx
>
>          movl    WAKEUP_CTX(mtrr), %eax
>
>          wrmsr
> system hangs, but if I insert it just after them - system reboots.
>
> With just commenting this three lines I was able to get successful
> suspend/resume with UP amd64 kernel!!!

Good catch!  I can confirm this is a correct bandaid.  We cannot 
restore this MSR without restoring entire MTRR map.  Actually, I 
should have written separate functions to save/restore all global 
MSRs.  Only per-CPU MSRs should be embedded like that.

> Here is problems I still have now:
>   - SMP kernel resume is not working, system reboots while doing
> acpi_wakeup_cpus();

My dual-core CPU seems to resume okay but quite unstable.  Can you try 
something like the following in amd64/mp_machdep.c and tell me if it 
helps?

------------
@@ -57,6 +57,7 @@
 #include <vm/vm_extern.h>

 #include <machine/apicreg.h>
+#include <machine/cpufunc.h>
 #include <machine/md_var.h>
 #include <machine/mp_watchdog.h>
 #include <machine/pcb.h>
@@ -1121,6 +1121,8 @@
	int cpumask = PCPU_GET(cpumask);
 
	if (savectx2(&stopxpcbs[cpu])) {
+		/* Flush CPU cache. */
+		wbinvd();
		/* Indicate that we are suspended. */
		atomic_set_int(&stopped_cpus, cpumask);
	} else {
------------

>   - text mode video does not restores on resume, while Xorg graphic
> one does. hw.acpi.reset_video=1 does not help, it just hanging
> resume process.

It is very common problem for modern video cards.  We cannot do much 
here without help of GPU-specific routines (e.g., ATI ATOM BIOS 
parser for RadeonHD) or in-kernel realmode emulation[1] (e.g., 
NetBSD).

Thanks for the feedback!

Jung-uk Kim

[1] http://cvsweb.netbsd.org/bsdweb.cgi/src/common/lib/libx86emu/
http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/arch/x86/x86/vga_post.c