Your suggestions about this Dell configuration?

[Frank Shute]

Hi VeeJay,

On Fri, May 23, 2008 at 06:24:42PM +0200, VeeJay wrote:
>
> Hello Frank
> 
> Really good points. I am really glad to have your thoughts. Regarding your
> questions and comments, I have given some answers and a couple of questions
> in *RED* colour. Please comment if you happen to manage some time during
> weekend, Thanks!
> 
> *Please continue...*
> 
> On Thu, May 22, 2008 at 4:01 AM, Frank Shute <frank at shute.org.uk> wrote:
> 
> > On Wed, May 21, 2008 at 08:49:51AM +0200, VeeJay wrote:
> > >
> > > Hello friends,
> > >
> > > My employer is buying this Dell server and I would like to have your
> > > opinion about the configuration.
> > >
> > > Requirements are:
> > > 2 Websites with 3-4 million hits per month with video ads.
> >
> > If it's "3-4 million hits per month" as you've stated twice now, then
> > your hardware is complete overkill.
> >
> > So I'll assume you mean 3-4 million hits a day for each site.
> 
> 
> *No, its 3-4 million each site per month and we are having problem. Because,
> either Apache or MySQL stops responding. I have following settings as
> Performance:*

If it's 3-4 million for each site per month, the hardware you are
proposing to buy is complete overkill. As an earlier poster said, he
managed that on a 486.

What hardware and OS are you presently using?

You have to determine what exactly is causing you problems and fix it.
Then you probably don't have to buy new hardware.


> **
> *# =================================================
> # Performance settings
> # =================================================
> Timeout 300
> KeepAlive On
> MaxKeepAliveRequests 100
> KeepAliveTimeout 15
> MinSpareServers 5
> MaxSpareServers 10
> StartServers 5
> MaxClients 256
> MaxRequestsPerChild 0*

<snip upgraded hardware specs>

> >
> > The performance of this hardware will depend on what *sort* of hits
> > you get. Are a lot of them just for the homepage? Then just cache it.
> >
> > Is it static content?
> 
> 
> *No, its dynamic contents, data is coming form Database.*
> 
> >
> >
> > If you're getting lots of ad-hoc database queries and fetches/writes
> > from/to disk, then your disks could get a thrashing.
> >
> > How big's your database? Being read from more than written to? How
> > precious is the data?
> >
> *More than 20 million records and more than 1000 Tables.*
> *And of course, data is always preciouse. :)*

Is this just one database or a number of databases? Whatever, at
a guess it looks like your data isn't normalised, although it depends
to a large extent what the data is. Do you have such a thing as an
entity/relationship diagram for your database(s)?

Did you get a DBA to design and implement your database(s)?

How long are your queries taking? If they're taking much more than a
second, then you are going to start having a backlog of non-returned
queries which are each tying up an httpd process, a PHP process and a
mysql process and your machine will struggle to keep up even with the
on average >1<2 hits per second.

The way to fix queries that are taking too long can be numerous but a
properly normalised database, cached regular queries and indexes can
all help. This is why you need a DBA or you have to subscribe to the
MySQL list and read the MySQL docs thoroughly - I believe the MySQL
docs touch on tuning. Although they don't tell you how to design or
normalise a database. You learn about that in Codd's and Date's books
I believe.

I learnt it at uni (but have largely forgotten it). Must read my
course materials again.

You could also have a locking problem i.e a table is locked for
writing (I don't think MySQL does row-level locking) and other
processes are being held up by the locked table.

> 
> >
> > How many of these hits are reading video ads? All of them? How many
> > KBs are these awful ads?
> >
> *50% of users are going to see the Video Ads.*
> **
> *Size would vary between 100KB to 2MB. *

Here, you've got a problem. I'm on a 80KB/s DSL which is probably
about average in the West, although there are plenty on dial-up. So to
download the "ad" it would take me anywhere from over a second to half
a minute. When I'm doing that, I'm tying up one of your Apache
processes. In that (average) half minute say, you're going to have 40
odd hits, 20 of them downloading ads. So you need at least 21 httpd
processes available. But (from above):

MaxSpareServers 10
StartServers 5

you've only got 15 available max!

But I wouldn't necessarily immediately increase the number of servers
available, you could make things worse. After all, your machine is
struggling to serve those 15. You have to find out why first. 

> 
> What bandwidth do you have to these servers?
> *100 Mbps*

20 x 80KB/s = 12800Kbps = 12.8 Mbps

You've got enough bandwidth.

> 
> How you are going to get the best out of your hardware depends on
> questions like these, so you have to analyse your Apache logs and tune
> appropriately.
> 
> Tuning Apache, mysql and PHP are all subjects in their own right.
> 
> For FreeBSD, read tuning(7).
> 
> Are you running FreeBSD ATM? Then some numbers from iostat, top etc.
> would be useful in analysing how your new server is going to cope and
> how much spare capacity you'll have, but the numbers are dependent on
> how you've tuned it (if at all).

Have a look at what I've mentioned in the paragraph above and look at
your logs: Apache and MySQL.

How about:

$ ps ax | grep httpd | wc -l

for a start to see how many servers you've got running.

Use the tools MySQL supply you with to test out and report on your
DB(s). Or just use the MySQL client to do common SQL queries and see
how long they take (it tells you).

What you want to do is replicate your server environment to your
workstation and start testing it first. As testing, tweaking or
re-implementing your DB on something that's live is not recommended.

Write some scripts etc. to hammer your DB/Apache on your workstation.
Do some timings and sums. Tweak various parameters, rewrite your PHP
queries, twiddle with your DB etc. until it performs.

You have to discover what your problem is. Just throwing hardware at
it might or might not work. Fixing your software stack will.

> 
> Hope I've given you something to think about.

Regards,

-- 

 Frank 

	
 Contact info: http://www.shute.org.uk/misc/contact.html