switching schedulers (Re: SCHED_ULE should not be the default)

[Luigi Rizzo]

On Fri, Dec 16, 2011 at 03:11:43AM +0100, C. P. Ghost wrote:
> On Thu, Dec 15, 2011 at 10:44 AM, Tom Evans <tevans.uk at googlemail.com> wrote:
> > Real time scheduler changing would be insane! I was thinking that
> > both/any/all schedulers could be compiled into the kernel, and the
> > choice of which one to use becomes a boot time configuration. You
> > don't have to recompile the kernel to change timecounter.
> 
> Right.
> 
> Switching the scheduler on the fly may be thinkable though.
> I could imagine a syscall that would suspend all scheduling,
> convert the bookkeeping data of one scheduler into the other
> scheduler's, and transfer control to the other scheduler. Of
> course, that would require some heavy hacking, as I would
> imagine that "cross-scheduler surgery" would result in a pretty
> hard to debug kernel (at least during development).

Since the subject has come up a few times:
back in 2002 (boy, it's almost 10 years ago!) we did implement
switchable schedulers on FreeBSD 4.x UP, and the diffs and
a bit of documentation are still online, and probably the architecture
could be reused even now or for the SMP case.

    announcement and brief description
	http://kerneltrap.org/node/349

    the patch referred in there
	http://info.iet.unipi.it/~luigi/ps_sched.20020719a.diff

The interesting part is probably the definition of the methods that
schedulers should implement (see struct _sched_interface ).

The switch from one scheduler to another was implemented with a
sysctl. This calls the sched_move() method of the current (i.e.
old) scheduler, which extracts all ready processes from its own
"queues" (however they are implemented) and reinserts them onto the
new scheduler's "queues" using its (new) setrunqueue() method.  You
don't need to bother for blocked process as the scheduler doesn't
know much about them.

I am not preserving the thread's dynamic "priority" (think of
accumulated work, affinity etc.) when switching
schedulers, as that is expected to be an infrequent event, and
so in the end it doesn't really matter -- at a switch, threads
are inserted in the scheduler as newly created ones, using only
the static priority as a parameter.

At the time I did not address the SMP case for several reasons,
but they are all gone now:
- did not have a suitable test system
- SMP support was still in a state of flux
- i did not understand the KSE concept
- and i did not have an algorithm for proportional share scheduling
  (the actual goal of the project) in an SMP context.

cheers
luigi

> 
> A more general solution could even be a separate userland
> scheduler process a la L4 [*], but since we don't have lightweight
> IPC in the kernel (yet, or never), it would require even heavier
> black wizardry. But nice and flexible it would be. ;-)
> 
> [*] Refs:
> - https://github.com/l4ka/pistachio
> - http://www.systems.ethz.ch/education/past-courses/fall-2010/aos/lectures/wk13-scheduling-print.pdf
> 
> Regards,
> -cpghost.
> 
> -- 
> Cordula's Web. http://www.cordula.ws/
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"