99% CPU usage in System (Was: Re: vinum in 4.x poor performer?)
Marc G. Fournier
scrappy at hub.org
Wed Feb 9 17:09:15 PST 2005
still getting this:
# vmstat 5
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr da0 da1 in sy cs us sy id
11 2 0 3020036 267944 505 2 1 1 680 62 0 0 515 4005 918 7 38 55
19 2 0 3004568 268672 242 0 0 0 277 0 0 3 338 2767 690 1 99 0
21 2 0 2999152 271240 135 0 0 0 306 0 6 9 363 1749 525 1 99 0
13 2 0 3001508 269692 87 0 0 0 24 0 3 3 302 1524 285 1 99 0
17 2 0 3025892 268612 98 0 1 0 66 0 5 6 312 1523 479 3 97 0
Is there a way of determining what is sucking up so much Sys time? stuff
like pperl scripts running and such would use 'user time', no? I've got
some high CPU processes running, but would expect them to be shooting up
the 'user time' ...
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
setiathome 21338 16.3 0.2 7888 7408 ?? RJ 9:05PM 0:11.35 /usr/bin/perl -wT /usr/local/majordomo/bin/mj_queuerun -v 0
setiathome 21380 15.1 0.1 2988 2484 ?? RsJ 9:06PM 0:02.42 /usr/bin/perl -wT /usr/local/majordomo/bin/mj_enqueue -r -d postgresql.org -l pgsql-sql -P10 -p10
setiathome 21384 15.5 0.1 2988 2484 ?? RsJ 9:06PM 0:02.31 /usr/bin/perl -wT /usr/local/majordomo/bin/mj_enqueue -r -d postgresql.org -l pgsql-docs -P10 -p10
setiathome 21389 15.0 0.1 2720 2216 ?? RsJ 9:06PM 0:02.06 /usr/bin/perl -wT /usr/local/majordomo/bin/mj_enqueue -r -d postgresql.org -l pgsql-hackers -P10 -p10
setiathome 21386 13.7 0.1 2720 2216 ?? RsJ 9:06PM 0:02.03 /usr/bin/perl -wT /usr/local/majordomo/bin/mj_enqueue -r -d postgresql.org -l pgsql-ports -P10 -p10
setiathome 21387 13.2 0.1 2724 2220 ?? RsJ 9:06PM 0:01.92 /usr/bin/perl -wT /usr/local/majordomo/bin/mj_enqueue -r -d postgresql.org -l pgsql-interfaces -P10 -p10
setiathome 21390 14.6 0.1 2724 2216 ?? RsJ 9:06PM 0:01.93 /usr/bin/perl -wT /usr/local/majordomo/bin/mj_enqueue -o -d postgresql.org -l pgsql-performance -P10 -p10
setiathome 21330 12.0 0.2 8492 7852 ?? RJ 9:05PM 0:15.55 /usr/bin/perl -wT /dev/fd/3//usr/local/www/mj/mj_wwwusr (perl5.8.5)
setiathome 7864 8.9 0.2 8912 8452 ?? RJ 7:20PM 29:54.88 /usr/bin/perl -wT /usr/local/majordomo/bin/mj_trigger -t hourly
Is there some way of finding out where all the Sys Time is being used?
Something more fine grained them what vmstat/top shows?
On Wed, 9 Feb 2005, Loren M. Lang wrote:
> On Wed, Feb 09, 2005 at 02:32:30AM -0400, Marc G. Fournier wrote:
>>
>> Is there a command that I can run that provide me the syscall/sec value,
>> that I could use in a script? I know vmstat reports it, but is there an
>> easier way the having to parse the output? a perl module maybe, that
>> already does it?
>
> vmstat shouldn't be too hard to parse, try the following:
>
> vmstat|tail -1|awk '{print $15;}'
>
> To print out the 15th field of vmstat. Now if you want vmstat to keep
> running every five seconds or something, it's a little more complicated:
>
> vmstat 5|grep -v 'procs\|avm'|awk '{print $15;}'
>
>>
>> Thanks ...
>>
>> On Wed, 9 Feb 2005, Marc G. Fournier wrote:
>>
>>> On Tue, 8 Feb 2005, Dan Nelson wrote:
>>>
>>>> Details on the array's performance, I think. Software RAID5 will
>>>> definitely have poor write performance (logging disks solve that
>>>> problem but vinum doesn't do that), but should have excellent read
>>>> rates. From this output, however:
>>>>
>>>>> systat -v output help:
>>>>> 4 users Load 4.64 5.58 5.77
>>>>
>>>>> Proc:r p d s w Csw Trp Sys Int Sof Flt
>>>>> 24 9282 949 8414***** 678 349 8198
>>>>
>>>>> 54.6%Sys 0.2%Intr 45.2%User 0.0%Nice 0.0%Idl
>>>>
>>>>> Disks da0 da1 da2 da3 da4 pass0 pass1
>>>>> KB/t 5.32 9.50 12.52 16.00 9.00 0.00 0.00
>>>>> tps 23 2 4 3 1 0 0
>>>>> MB/s 0.12 0.01 0.05 0.04 0.01 0.00 0.00
>>>>> % busy 3 1 1 1 0 0 0
>>>>
>>>> , it looks like your disks aren't being touched at all. You are doing
>>>> over 99999 syscalls/second, though, which is mighty high. The 50% Sys
>>>> doesn't look good either. You may have a runaway process doing some
>>>> syscall over and over. If this is not an MPSAFE syscall (see
>>>> /sys/kern/syscalls.master ), it will also prevent other processes from
>>>> making non-MPSAFE syscalls, and in 4.x that's most of them.
>>>
>>> Wow, that actually pointed me in the right direction, I think ... I just
>>> killed an http process that was using alot of CPU, and syscalls drop'd
>>> down to a numeric value again ... I'm still curious as to why this only
>>> seem sto affect my Dual-Xeon box though :(
>>>
>>> Thanks ...
>>>
>>> ----
>>> Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
>>> Email: scrappy at hub.org Yahoo!: yscrappy ICQ: 7615664
>>> _______________________________________________
>>> freebsd-questions at freebsd.org mailing list
>>> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
>>> To unsubscribe, send any mail to
>>> "freebsd-questions-unsubscribe at freebsd.org"
>>>
>>
>> ----
>> Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
>> Email: scrappy at hub.org Yahoo!: yscrappy ICQ: 7615664
>> _______________________________________________
>> freebsd-questions at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
>> To unsubscribe, send any mail to "freebsd-questions-unsubscribe at freebsd.org"
>
> --
> I sense much NT in you.
> NT leads to Bluescreen.
> Bluescreen leads to downtime.
> Downtime leads to suffering.
> NT is the path to the darkside.
> Powerful Unix is.
>
> Public Key: ftp://ftp.tallye.com/pub/lorenl_pubkey.asc
> Fingerprint: B3B9 D669 69C9 09EC 1BCD 835A FAF3 7A46 E4A3 280C
>
>
----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy at hub.org Yahoo!: yscrappy ICQ: 7615664
More information about the freebsd-questions
mailing list