Sendmail and high kernel CPU utilisation
Travis Fitch
travis at fitch.id.au
Wed Jun 21 07:37:11 UTC 2006
Travis Fitch wrote:
> Hello,
>
> I was hoping someone might be able to shed some light on an issue I am
> having with sendmail on FreeBSD 5.2.1 running on a Sun V120.
>
> If you look at the snippet from top you can see that several sendmail
> process are casing the kernel to use ~ 65% of the CPU. I am having some
> issues profiling what is causing this issue. You can also see the load is
> quite high.
>
> last pid: 72705; load averages: 12.22, 12.29, 12.00
> up 1+06:09:44 11:42:28
> 86 processes: 14 running, 72 sleeping
> CPU states: 35.5% user, 0.0% nice, 64.5% system, 0.0% interrupt, 0.0%
> idle
> Mem: 148M Active, 104M Inact, 73M Wired, 520K Cache, 60M Buf, 131M Free
> Swap: 2057M Total, 102M Used, 1954M Free, 4% Inuse
>
> PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND
> 20958 root 122 0 23368K 5472K RUN 44:32 8.64% 8.64% sendmail
> 62864 root 122 0 23432K 5528K RUN 10:45 8.54% 8.54% sendmail
> 70522 root 122 0 23448K 5544K RUN 2:43 8.35% 8.35% sendmail
> 91279 smmsp 122 0 12224K 3832K RUN 60:17 8.25% 8.25% sendmail
> 66302 root 122 0 23392K 5472K RUN 8:36 8.25% 8.25% sendmail
> 16850 root 122 0 23432K 5528K RUN 51:31 8.20% 8.20% sendmail
> 66306 root 121 0 23448K 5528K RUN 8:34 8.11% 8.11% sendmail
> 68330 root 121 0 23432K 5528K RUN 5:37 8.11% 8.11% sendmail
> 51654 root 121 0 23392K 5472K RUN 16:40 8.06% 8.06% sendmail
> 66377 root 121 0 23392K 5472K RUN 8:22 8.01% 8.01% sendmail
> 69364 root 121 0 23432K 5504K RUN 3:50 8.01% 8.01% sendmail
>
>
> The other interesting thing is that a lot of the SMTP connections hang
> around for quite some time. These connection just keep building up and
> slowly bring the system to a crawl.
>
> correo:root# ps aux| grep -v grep | grep sendmail
> root 62864 8.0 1.1 23432 5528 ?? R 10:06AM 10:49.58 sendmail:
> k5J06CMN062864 mx02.globalcenter.net.au [203.89.192.35]: DATA (sendmail)
> root 66306 8.0 1.1 23448 5528 ?? R 10:18AM 8:38.72 sendmail:
> k5J0IoxC066306 omta02ps.mx.bigpond.com [144.140.83.154]: DATA (sendmail)
> root 66377 7.9 1.1 23392 5472 ?? R 10:20AM 8:26.25 sendmail:
> k5J0KHGa066377 vpn02.commandhub.net [203.89.202.122]: DATA (sendmail)
> root 20958 8.0 1.1 23368 5472 ?? R 5:14AM 44:36.44 sendmail:
> k5IJF2bd020958 [83.173.162.41]: DATA (sendmail)
> root 68330 8.0 1.1 23432 5528 ?? R 10:43AM 5:41.33 sendmail:
> k5J0hOAk068330 mx01.globalcenter.net.au [203.89.192.34]: DATA (sendmail)
> root 69364 8.0 1.1 23432 5504 ?? R 10:59AM 3:54.45 sendmail:
> k5J0xmg6069364 omta05ps.mx.bigpond.com [144.140.83.195]: DATA (sendmail)
> root 70522 8.0 1.1 23448 5544 ?? R 11:11AM 2:47.79 sendmail:
> k5J1BY7Y070522 mx01.globalcenter.net.au [203.89.192.34]: DATA (sendmail)
> root 66302 8.0 1.1 23392 5472 ?? R 10:18AM 8:40.20 sendmail:
> k5J0If8t066302 vpn02.commandhub.net [203.89.202.122]: DATA (sendmail)
> smmsp 91279 8.0 0.7 12224 3832 ?? Rs 4:01AM 60:20.92 sendmail:
> Queue runner at 00:10:00 for /var/spool/clientmqueue (sendmail)
> root 16850 8.0 1.1 23432 5528 ?? R 4:30AM 51:34.91 sendmail:
> k5IIUaWm016850 [201.150.67.51]: DATA (sendmail)
> root 51654 8.0 1.1 23392 5472 ?? R 9:37AM 16:44.52 sendmail:
> k5INbbYo051654 vpn02.commandhub.net [203.89.202.122]: DATA (sendmail)
> root 91276 0.0 1.0 22928 5032 ?? Ss 4:01AM 0:03.05 sendmail:
> rejecting connections on daemon MSA: load average: 12 (sendmail)
>
> I have rebuilt the world and kernel to see if the resolves my issue, but
> alas no luck.
>
> Hopefully someone will be able to point be in the right direction.
>
> Regards,
>
> Travis
>
> _______________________________________________
> freebsd-questions at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "freebsd-questions-unsubscribe at freebsd.org"
>
Further to this, when I put truss or ktrace into action on one of the
sendmail process that is using a lot of kernel time, I see the following
gettimeofday(0x40dfc210,0x0) = 0 (0x0)
poll(0x30e000,0x1,0xffffffffffffffff) = 1 (0x1)
gettimeofday(0x40dfc210,0x0) = 0 (0x0)
poll(0x30e000,0x1,0xffffffffffffffff) = 1 (0x1)
gettimeofday(0x40dfc210,0x0) = 0 (0x0)
poll(0x30e000,0x1,0xffffffffffffffff) = 1 (0x1)
gettimeofday(0x40dfc210,0x0) = 0 (0x0)
poll(0x30e000,0x1,0xffffffffffffffff) = 1 (0x1)
gettimeofday(0x40dfc210,0x0) = 0 (0x0)
poll(0x30e000,0x1,0xffffffffffffffff) = 1 (0x1)
gettimeofday(0x40dfc210,0x0) = 0 (0x0)
poll(0x30e000,0x1,0xffffffffffffffff) = 1 (0x1)
gettimeofday(0x40dfc210,0x0) = 0 (0x0)
poll(0x30e000,0x1,0xffffffffffffffff) = 1 (0x1)
gettimeofday(0x40dfc210,0x0) = 0 (0x0)
poll(0x30e000,0x1,0xffffffffffffffff) = 1 (0x1)
gettimeofday(0x40dfc210,0x0) = 0 (0x0)
poll(0x30e000,0x1,0xffffffffffffffff) = 1 (0x1)
gettimeofday(0x40dfc210,0x0) = 0 (0x0)
poll(0x30e000,0x1,0xffffffffffffffff) = 1 (0x1)
gettimeofday(0x40dfc210,0x0) = 0 (0x0)
I am not running ntpd atm, however ntpdate to pool.ntp.org syncs ok.
Does, this help shed some light?
Regards,
Travis
More information about the freebsd-questions
mailing list