amrd disk performance drop after running under high load
christias at gmail.com
Sun Nov 11 17:06:44 PST 2007
On Nov 11, 2007 7:26 PM, Alexey Popov <lol at chistydom.ru> wrote:
> Kris Kennaway wrote:
> >>> In the "good" case you are getting a much higher interrupt rate but
> >>> with the data you provided I can't tell where from. You need to run
> >>> vmstat -i at regular intervals (e.g. every 10 seconds for a minute)
> >>> during the "good" and "bad" times, since it only provides counters
> >>> and an average rate over the uptime of the system.
> >> Now I'm running 10-process lighttpd and the problem became no so big.
> >> I collected interrupt stats and it shows no relation beetween
> >> ionterrupts and slowdowns. Here is it:
> >> http://22.214.171.124/gprof/intr-graph/
> >> Also I have similiar statistics on mutex profiling and it shows
> >> there's no problem in mutexes.
> >> http://126.96.36.199/gprof/mtx-graph/mtxgifnew/
> >> I have no idea what else to check.
> > I don't know what this graph is showing me :) When precisely is the
> > system behaving poorly?
> Take a look at "Disk Load %" picture at
> At ~ 17:00, 03:00-04:00, 13:00-14:00, 00:30-01:30, 11:00-13:00 it shows
> peaks of disk activity which really never happen. As I said in the
> beginning of the thread in this "peak" moments disk becomes slow and
> vmstat shows 100% disk load while performing < 10 tps. Other grafs at
> this page shows that there's no relation to interrupts rate of amr or em
> device. You advised me to check it.
> When I was using single-process lighttpd the problem was much harder as
> you can see at http://188.8.131.52/gprof/graph/ . At first picture on
> this page you can see disk load peaks at 18:00 and 15:00 which leaded to
> decreasing network output because disk was too slow.
> Back in this thread we suspected UMA mutexes. In order to check it I
> collected mutex profiling stats and draw graphs over time and they also
> didn't show anything interesting. All mutex graphs were smooth while
> disk load peaks. http://184.108.40.206/gprof/mtx-graph/mtxgifnew/
> With best regards,
> Alexey Popov
what is your RAID controller configuration (read ahead/cache/write
policy)? I have seen weird/bogus numbers (~100% busy) reported by
systat -v when read ahead was enabled on LSI/amr controllers.
freebsd-hackers at freebsd.org mailing list
To unsubscribe, send any mail to "freebsd-hackers-unsubscribe at freebsd.org"
More information about the freebsd-stable