SCHED_ULE should not be the default

Oliver Pinter oliver.pntr at gmail.com
Thu Dec 15 07:41:13 UTC 2011


On 12/15/11, Jeremy Chadwick <freebsd at jdc.parodius.com> wrote:
> On Thu, Dec 15, 2011 at 03:05:12AM +0100, Oliver Pinter wrote:
>> On 12/15/11, O. Hartmann <ohartman at zedat.fu-berlin.de> wrote:
>> > On 12/14/11 18:54, Tom Evans wrote:
>> >> On Wed, Dec 14, 2011 at 11:06 AM, George Mitchell
>> >> <george+freebsd at m5p.com> wrote:
>> >>>
>> >>> Dear Secret Masters of FreeBSD: Can we have a decision on whether to
>> >>> change back to SCHED_4BSD while SCHED_ULE gets properly fixed?
>> >>>
>> >>
>> >> Please do not do this. This thread has shown that ULE performs poorly
>> >> in very specific scenarios where the server is loaded with NCPU+1 CPU
>> >> bound processes, and brought forward more complaints about
>> >> interactivity in X (I've never noticed this, and use a FreeBSD desktop
>> >> daily).
>> >
>> > I would highly appreciate a decission against SCHED_ULE as the default
>> > scheduler! SCHED_4BSD is considered a more mature entity and obviously
>> > it seems that SCHED_ULE needs some refinements to achieve a better level
>> > of quality.
>> >
>> >>
>> >> On the other hand, we have very many benchmarks showing how poorly
>> >> 4BSD scales on things like postgresql. We get much more load out of
>> >> our 8.1 ULE DB and web servers than we do out of our 7.0 ones. It's
>> >> easy to look at what you do and say "well, what suits my environment
>> >> is clearly the best default", but I think there are probably more
>> >> users typically running IO bound processes than CPU bound processes.
>> >
>> > You compare SCHED_ULE on FBSD 8.1 with SCHED_4BSD on FBSD 7.0? Shouldn't
>> > you compare SCHED_ULE and SCHED_4BSD on the very same platform?
>> >
>> > Development of SCHED_ULE has been focused very much on DB like
>> > PostgreSQL, no wonder the performance benefit. But this is also a very
>> > specific scneario where SCHED_ULE shows a real benefit compared to
>> > SCHED_4BSD.
>> >
>> >>
>> >> I believe the correct thing to do is to put some extra documentation
>> >> into the handbook about scheduler choice, noting the potential issues
>> >> with loading NCPU+1 CPU bound processes. Perhaps making it easier to
>> >> switch scheduler would also help?
>> >
>> > Many people more experst in the issue than myself revealed some issues
>> > in the code of both SCHED_ULE and even SCHED_4BSD. It would be a pitty
>> > if all the discussions get flushed away like a "toilette-busisness" as
>> > it has been done all the way in the past.
>> >
>> >
>> > Well, I'd like to see a kind of "standardized" benchmark. Like on
>> > openbenchmark.org or at phoronix.com. I know that Phoronix' way of
>> > performing benchmarks is questionable and do not reveal much of the
>> > issues, but it is better than nothing. I'm always surprised by the worse
>> > performance of FreeBSD when it comes to threaded I/O. The differences
>> > between Linux and FreeBSD of the same development maturity are
>> > tremendous and scaring!
>> >
>> > It is a long time since I saw a SPEC benchmark on a FreeBSD driven HPC
>> > box. Most benchmark around for testing hardware are performed with Linux
>> > and Linux seems to make the race in nearly every scenario. It would be
>> > highly appreciable and interesting to see how Linux and FreeBSD would
>> > perform in SPEC on the same hardware platform. This is only an idea.
>> > Without a suitable benchmark with a codebase understood the discussion
>> > is in many aspects pointless -both ways.
>> >
>> >
>> >>
>> >> Cheers
>> >>
>> >> Tom
>> >>
>> >> References:
>> >>
>> >> http://people.freebsd.org/~kris/scaling/mysql-freebsd.png
>> >> http://suckit.blog.hu/2009/10/05/freebsd_8_is_it_worth_to_upgrade
>> >> _______________________________________________
>>
>> Hi!
>>
>> Can you try with this settings:
>> op at opn ~> sysctl kern.sched.
>
> I'm replying with a list of each setting which differs compared to
> RELENG_8 stock on our ULE systems.  Note that our ULE systems are 1
> physical CPU with 4 cores.

On other system that has 4 core I use 7-STABLE, because I have not
enough time for upgraded it, and the system has some custom patches.
The  values what I send in previous mail mostly based on this 4 cores
system.

>
>> kern.sched.cpusetsize: 8
>
> I see no such tunable/sysctl on any of our RELENG_8 and RELENG_7
> systems.  Nor do I find any references to it in /usr/src (on any
> system).  Is this a RELENG_9 setting?  Please explain where it comes
> from.  I hope it's not a custom kernel patch...

Yes, this is 9-STABLE.

>
>> kern.sched.preemption: 0
>
> This differs; default value is 1.

PREEMPTION is disabled via kernel config.

>
>> kern.sched.name: ULE
>> kern.sched.slice: 13
>> kern.sched.interact: 30
>
>> kern.sched.preempt_thresh: 224
>
> This differs; default value is 64.  The "magic value" of 224 has been
> discussed in the past, in this thread even.

This magic value has discussed before 1 or 1.5 year here, first for 8-STABLE.

>
>> kern.sched.static_boost: 152
>
> This differs; on our systems it's 160.
>
>> kern.sched.idlespins: 10000
>
>> kern.sched.idlespinthresh: 16
>
> This differs; on our systems it's 4.
>
>> Most of them from 7-STABLE settings, and with this, "works for me".
>> This an laptop with core2 duo cpu (with enabled powerd), and my kernel
>> config is here:
>> http://oliverp.teteny.bme.hu/freebsd/kernel_conf
>
> --
> | Jeremy Chadwick                                jdc at parodius.com |
> | Parodius Networking                       http://www.parodius.com/ |
> | UNIX Systems Administrator                   Mountain View, CA, US |
> | Making life hard for others since 1977.               PGP 4BD6C0CB |
>
>


More information about the freebsd-stable mailing list