suspending threads before devices [Was: svn commit: r233249 - head/sys/amd64/acpica]

[Andriy Gapon]

On 22/03/2012 16:14, Konstantin Belousov wrote:
> I already noted this to Jung-uk, I think that current suspend handling
> is (somewhat) wrong. We shall not stop other CPUs for suspension when
> they are executing some random kernel code. Rather, CPUs should be safely
> stopped at the kernel->user boundary, or at sleep point, or at designated
> suspend point like idle loop.
> 
> We already are engaged into somewhat doubtful actions like restoring of %cr2,
> since we might, for instance, preemt page fault handler with suspend IPI.

I recently revisited this issue in the context of some suspend+resume problems
that I am having with radeonkms driver.  What surprised me is that the driver's
suspend code has no synchronization whatsoever with its other code paths.  So, I
looked first at the Linux code and then at the illumos code to see how suspend
is implemented there.
As far as I can see, those kernels do exactly what you suggest that we do.
Before suspending devices they first suspend all threads except for one that
initiates the suspend.  For userland threads a signal-like mechanism is used to
put them in a state similar to SIGSTOP-ed one.  With the kernel threads
mechanisms are different between the kernels.  Also, illumos freezes kernel
threads after suspending the devices, not before.

I think that we could start with only the userland threads initially.  Do you
think the SIGSTOP-like approach would be hard to implement for us?

References:

http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/cpr/cpr_main.c#425
http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/cpr/cpr_uthread.c#80

http://lxr.free-electrons.com/source/kernel/power/suspend.c#L388
http://lxr.free-electrons.com/source/kernel/power/suspend.c#L207
http://lxr.free-electrons.com/source/kernel/power/power.h#L235
http://lxr.free-electrons.com/source/kernel/power/process.c#L118
http://lxr.free-electrons.com/source/kernel/power/process.c#L27
http://lxr.free-electrons.com/source/kernel/freezer.c#L115

-- 
Andriy Gapon