amd64/174409: stopping during resume caused by io_apic.c change(r243764)

[Andriy Gapon]

The following reply was made to PR amd64/174409; it has been noted by GNATS.

From: Andriy Gapon <avg at FreeBSD.org>
To: KAHO Toshikazu <kaho at elam.kais.kyoto-u.ac.jp>
Cc: bug-followup at FreeBSD.org
Subject: Re: amd64/174409: stopping during resume caused by io_apic.c change(r243764)
Date: Wed, 26 Dec 2012 09:14:33 +0200

 on 26/12/2012 05:27 KAHO Toshikazu said the following:
 >   Hello,
 > 
 >> Could you please re-apply r243764 and then try the following _hack_?
 > 
 >   Yes, the machine can wake up with pure head src + this patch.
 
 Here is my current understanding of the situation.
 r243764 is actually an improvement, not a regression.
 
 What happens without r243764 during resume: ioapic pin 9 which is used for
 SCI/IRQ9 has some garbage in its CPU destination configuration, we unmask the
 pin, ioapic sends interrupt message to nowhere, the interrupt is never accepted
 and so the pin is stuck even after we program it with the correct destination.
 This is bad.
 
 What happens with r243764 during resume: we first program the correct
 destination (which happens to be BSP), then unmask the pin; as soon as
 interrupts on the BSP are re-enable SCI/IRQ9 is correctly delivered.
 This is the correct behavior, IMO.
 But apparently there is some problem in our ACPI code which leads to the hang.
 This is bad.
 
 While now I know where to look for the root cause, I don't know to do that in
 the early post-resume environment.  As I understand, there is no way to output
 any information from kernel.  This makes things hard.
 
 Some things to try (independently of each other) without the hack and with r243764:
 o cpuset -l 1 -x 9 -- to try to see what happens if SCI/IRQ9 is delivered to the
 AP instead of BSP
 
 o the following patch:
 --- a/sys/dev/acpica/acpi.c
 +++ b/sys/dev/acpica/acpi.c
 @@ -2776,6 +2776,9 @@ acpi_EnterSleepState(struct acpi_softc *sc, int state)
  	sleep_result = acpi_sleep_machdep(sc, state);
  	acpi_wakeup_machdep(sc, state, sleep_result, 0);
  	AcpiLeaveSleepStatePrep(state);
 +
 +	AcpiDisableAllGpes();
 +
  	intr_restore(intr);
 
  	/* call acpi_wakeup_machdep() again with interrupt enabled */
 
 >> Also, not sure if I asked before the following questions related to
 >> debugging possibilities.
 >> Does this system have a serial port?  Are you able to setup serial
 >> console access?
 >> Or the same for firewire port/console.
 >> Does this system preserve memory content across reboot/reset?
 >> In other words, can you see messages from the previous boot in dmesg
 >> after a reboot?
 > 
 >   The machine I'm using is a cheap mobile notebook with a ULV CPU.
 > It does not have a serial port and a firewire. I have USB-serial
 > adapters, but I don't think it helps to make a serial console.
 > I can only take a memo by a pen, if the machine does not reboot
 > immediately and the keyboard on DDB is alive.
 > 
 
 I see.
 
 -- 
 Andriy Gapon