some inter-cpu interaction changes for review
avg at FreeBSD.org
Tue May 10 12:46:20 UTC 2011
I've created a new branch user/avg/xcpu where I committed a number of changes that
I would like to propose for the main tree. The changes affect CPU stopping, both
hard (for panic and kdb) and soft (for suspend, reboot, etc); CPU
rendezvous/cross-calling; panic; kdb; system reset; TLB shootdowns for x86. MD
changes are provided only for amd64 at the moment (i386 should be trivial to add).
Some highlights of the changes:
o hard stopping is completely separated from soft stopping
Hard stopping is performed in a limited number of very special cases.
It also should have an ability to "penetrate" (almost) any contexts on CPUs being
o new approach to avoid inter-cpu deadlocks
The idea is that if a CPU waits either (a) on a lock protecting some inter-CPU
actions or (b) for other CPU to perform some action request by the master CPU,
then the waiting CPU should also check for incoming events while spinning.
For example, if two CPUs try to hardstop all other CPUs (and each other), then the
CPU that failed to acquire a hardstop lock should check if it's being hardstopped
while spinning on the lock. Similarly, when a CPU requests an action to be
performed on a set of CPUs and it waits for the action to be completed by all the
CPUs in the set via the rendezvous mechanism, then that CPU should check for
incoming rendezvous requests while spin-waiting.
o rendezvous actions can be done in parallel
Multiple rendezvous requests from different CPUs can be piggy-backed onto the same
IPI to a target CPU.
o x86 TLB shootdowns are re-implemented via the rendezvous mechanism
Please see the branch for all the changes and details.
Please review and/or test (on amd64 only) these changes.
I will appreciate any comments, suggestions, critique and help.
Especially the help with MD bits for other architectures iff you find the
suggested approach to be correct and useful.
P.S. I see the largeSMP work by Attilio and I think that those changes should go
in first and then I will adapt my changes for cpumask_t -> cpuset_t transition.
That is, I do not want to cause any delays for Attilio's work.
More information about the freebsd-arch