sched_ule performance on single CPU

[Unga]

--- "O. Hartmann" <ohartman at mail.zedat.fu-berlin.de>
wrote:

> Hello,
> 
> I experience also a strange lagg when using
> SCHED_ULE and FreeBSD 7.0 on 
> AMD64 platforms with and without UP. I tried to
> track on FreeBSD 7 from 
> the very early days, so I noticed some performance
> impacts last year 
> when something chenged in the scheduling. I'm not
> very familiar with the 
> differences, changes and changes in paradigm when
> 7.0 was introduced, 
> but the experience of massive lagging and slowing
> down the box when 
> either heavy SATA IO and network IO is performed is
> aware since then.
> At this very moment I use a private AMD64 box, a P4
> box and a Q6600 
> (QuadCore) box, all with very similar kernel
> configurations, if 
> possible. The AMD64 box is based on AMDs single core
> CPU 3500+ at 2,2 
> GHz running a UP (64bit) kernel, the P4 is a SMT
> capable CPU running a 
> SMP kernel (32bit) and so the more modern Q6600 quad
> core box (64bit). 
> All of them running FreeBSD 7.0 with most recent
> builds.
> On the SMP capable boxes I still recognize lagging
> and getting stuck 
> when building world and having also X11 running with
> some applications 
> like Firefox. But due to the performance of the quad
> core box this isn't 
> obvious in most times. On the 32Bit box with
> hyperthreading I see this 
> performance drops/laggs also, but not as massively
> as I relaize this on 
> the potentially faster UP, 64Bit box with UP kernel.
> On the pure 64bit 
> UP box, desktop get stuck for nearly a minute when
> doing a buildworld 
> (make -j2 or make -j1 doesn't matter, make -j4 gets
> worse). This 
> performance impact is also noticable when doing
> heavy network I/O, the 
> throughput does not even drops as expected, it gets
> stuck and stops 
> completely for some seconds.
> 
> This behaviour has been discussed on several German
> mailing lists and it 
> seems it is still present.
> 
> As far as I can say, with 32bit FreeBSD and with
> 6.3, all my boxes seem 
> to be more responsive as with 7.0, but the overall
> performance is better 
> in 7.0. But I fell really uncomfortable with getting
> stuck while under 
> heavy load.
> I will test this weird behaviour next under NFS
> conditions, using both 
> the UP and SMP boxes under heavy load as NFS servers
> for critical video 
> streams of some scientific datasets. If this
> behaviour is also impacting 
> NFS, I will report this here, again. But I guess as
> the other reports of 
> this misbehaviour will vanish in the darkness of the
> net ...
> 

Hi Oliver

Thanks for the detailed observations.

You mentioned you have following hardware:
1. The AMD64 box is based on AMDs single core CPU
3500+ at 2,2GHz running a UP (64bit) kernel

2. The P4 is a SMP capable CPU running a SMP kernel
(32bit)

3. The more modern Q6600 quad core box (64bit)


What is the speed of the P4 CPU? how many cores?

What is speed of the Q6600?

Btw, I don't have AMD64 machine, therefore, I cannot
comment on the implementation of  FreeBSD on AMD64,
but I can tell little bit more about scheduling, may
be useful to you.

Scheduling is serving. When you serve if there is
delay, lets see why.

Lets take a small example. Say we have N number of
guests (in computing, processes) at a table and one
waiter (in computing, the CPU) to serve.

If the N is small, ie. you have few guests (in
computing, less load) and the food is ready on a
nearby table (in computing, its in RAM), the waiter
can serve all without nobody feeling a delay.

If the N is large, (in computing, heavy load), the
food is ready on a nearby table (in computing, its in
RAM), the waiter is Olympic runner (in computing,
faster CPU), the waiter can still serve all without
nobody feeling a delay. But if the waiter is old and
walk slow, some may feel delay.

Even if our waiter is faster, if the food not on the
table and for every serve he has to cross the road to
get food (in computing, swapping), almost all may feel
the delay.

Its obvious If the there are multiple waiters (in
computing, multi core), nobody may feel a delay.

So to get better performance, you need a faster CPU or
multi cores. To avoid swapping, you need sufficient
RAM for job and you should not over load the machine.

Once you meet all the preconditions for performance
and you still don't get satisfactory performance, its
most probably the scheduler algorithm may not be good.

In FreeBSD, the guests on the table aka the processes
are divided into four categories by their priority.
They are: interrupt, realtime, normal and idle.

When an interrupt priority process require service, it
preempt all other lower priority processes and execute
the interrupt priority process. I/O operations,
swapping, etc. run in interrupt priority. 

You mentioned "experience of massive lagging and
slowing down the box when either heavy SATA IO and
network IO is performed is aware since then." In this
kind of situations, where I/O processing is too much
for the current speed of the CPU, you can either
increase the speed of the CPU, or introduce multi core
or best offload the I/O processing to a I/O
co-processor (hardware-based RAID, Intel PRO/1000
network cards can offload IP processing load, etc). 

I mentioned in my previous email, I listen to music
and browse at the same time when buildworld is
running. The buildworld is not a I/O intensive
process, it's a CPU intensive process. As long as I
provide sufficient CPU power and RAM, both 4BSD and
ULE schedulers perform well to the extent, the user of
the computer is not aware of such a buildworld process
is going on.

The point I mentioned in my original post is a
comparison of the 4BSD to the ULE scheduler. I
mentioned the desktop responsiveness of the 4BSD seems
better. But the ULE scheduler included in the FreeBSD
7.x is not the latest. The latest ULE scheduler is
included in the up coming FreeBSD 8.0.

Kind regards
Unga




      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ