Purchasing the correct hardware: dual-core intel? Big cache?

Wed Apr 26 07:10:35 UTC 2006

Bill

start with looking the DB design - I;ve had batch runs go from 5 days down
to 1/2 day purely by tuning the SQL. I'm not kidding, all the  DB tuning
quides say look at the SQL/design first, then look at hardware like
disk/data layout, then finally the CPU.

There's lots of info about tuning on Postgress, use this, you'll get more
out of the app this way than spending money chucking hardware at it.

--
martin

On 4/25/06, Bill Moran <wmoran at collaborativefusion.com> wrote:
>
> On Tue, 25 Apr 2006 13:28:50 +0100
> "Martin Hepworth" <maxsec at gmail.com> wrote:
> > Bill
> >
> > if the database is CPU dependant I'd look at tuning the queries/indexes
> and
> > that stuff...it really shouldn't be CPU bound.
>
> That's in progress, and it's going to be an ongoing process as the
> application goes through versions.
>
> Fact is, with 4G of RAM most of the data sits in RAM, so reads incur
> no or little IO.  With high-end SCSI disks in RAID-10 and a battery-
> backed cache, burst writes are cached, thus lightening fast, and
> we've been unable to run the application hard enough to saturate
> the SCSI bus so far.
>
> So ... the current bottleneck is CPU.
>
> >
> > --
> > martin
> >
> > On 4/24/06, Bill Moran <wmoran at potentialtech.com> wrote:
> > >
> > > On Mon, 24 Apr 2006 23:03:59 +0100
> > > "Martin Hepworth" <maxsec at gmail.com> wrote:
> > >
> > > > Bill
> > > >
> > > > depends on the application itself, but more RAM and the disk layout
> > > (RAID)
> > > > will be more important than the CPU. Also depends on how write-heavy
> the
> > > > apps are...
> > >
> > > Thanks for the feedback, Martin.
> > >
> > > I'm fully aware of the app-dependency - what I'm looking for is a way
> > > to test the application.  I've got 3 different clusters available for
> > > testing, but I'm not sure how to tell if the cache is getting used
> > > heavily or not.
> > >
> > > I've already determined that the database server is CPU-bound under
> > > our test load.  With high-speed SCSI disks and battery-backed RAID,
> > > there's not enough IO to stress the disk subsystem.  RAM is almost a
> > > non-issue.  With the machine stressed at full load, it's only using
> > > 1/8 of the available RAM.
> > >
> > > So, my current bottleneck is CPU power.  And the boss has asked me
> > > for the best way to overcome this bottleneck.  We're looking at either
> > > the same CPUs we already have, but with _huge_ caches (8m) - or going
> > > with more CPUs by getting true dual-core pentiums.
> > >
> > > The question this all pivots on is will 8M of cache be a significant
> > > improvement?  If not, then we're going with the dual-core CPUs.  What
> > > I'd like is some way to take an existing system and determine how
> often
> > > the cache is getting invalidated, so I can make some guesstemate as to
> > > whether more cache will help or not.
> > >
> > > >
> > > > --
> > > > martin
> > > >
> > > > On 4/24/06, Bill Moran <wmoran at potentialtech.com> wrote:
> > > > >
> > > > >
> > > > > I've been asked to make some hardware recommendations, I'm hoping
> some
> > > > > folks on the list can make some suggestions.
> > > > >
> > > > > We're looking hard at getting either Intel dual-core procs, or
> getting
> > > > > hyperthreaded procs with huge (8M) caches.
> > > > >
> > > > > We currently have a few dual proc Intel HT machines that we can
> test
> > > > > out our workload on, and I'm trying to get a feel for how to
> determine
> > > > > if a larger cache size will generate better performance than
> replacing
> > > > > HT procs with full-blown dual-core procs.  We're looking at the
> 6850
> > > > > from Dell, which supports both processor families:
> > > > >
> > > > >
> > >
> http://www1.us.dell.com/content/products/productdetails.aspx/pedge_6850?c=us&cs=555&l=en&s=biz
> > > > >
> > > > > The goal for these machines is to serve out PosgreSQL databases to
> as
> > > > > many Apache+php front ends as we can hang off each one.  So we're
> > > trying
> > > > > to purchase hardware that will create a DB server that can handle
> a
> > > lot
> > > > > of web server front ends.
> > > > >
> > > > > I have a Dell 2850 (dual HT procs) here that I can use for
> testing.
> > > > > I'm a little fuzzy on determining how well the cache is working,
> so
> > > I'm
> > > > > stuck on whether or not the 8M cache that's available on the HT
> units
> > > > > is worth the money or not.  Can anyone suggest a testing
> methodology
> > > > > that will isolate this particular aspect?
> > > > >
> > > > > --
> > > > > Bill Moran
> > > > > Collaborative Fusion Inc.
> > > > > _______________________________________________
> > > > > freebsd-questions at freebsd.org mailing list
> > > > > http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> > > > > To unsubscribe, send any mail to "
> > > > > freebsd-questions-unsubscribe at freebsd.org"
> > > > >
> > > > _______________________________________________
> > > > freebsd-questions at freebsd.org mailing list
> > > > http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> > > > To unsubscribe, send any mail to "
> > > freebsd-questions-unsubscribe at freebsd.org"
> > > >
> > > > *
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> ************************************************************************************
> > > > This footnote confirms that this email message has been scanned by
> > > > PineApp Mail-SeCure for the presence of malicious code, vandals &
> > > computer viruses.
> > > >
> > >
> ************************************************************************************
> > > >
> > > >
> > >
> > >
> > > --
> > > Bill Moran
> > > Collaborative Fusion Inc.
> > >
> > > ****************************************************************
> > > IMPORTANT: This message contains confidential information and is
> > > intended only for the individual named. If the reader of this
> > > message is not an intended recipient (or the individual
> > > responsible for the delivery of this message to an intended
> > > recipient), please be advised that any re-use, dissemination,
> > > distribution or copying of this message is prohibited. Please
> > > notify the sender immediately by e-mail if you have received
> > > this e-mail by mistake and delete this e-mail from your system.
> > > E-mail transmission cannot be guaranteed to be secure or
> > > error-free as information could be intercepted, corrupted, lost,
> > > destroyed, arrive late or incomplete, or contain viruses. The
> > > sender therefore does not accept liability for any errors or
> > > omissions in the contents of this message, which arise as a
> > > result of e-mail transmission.
> > > ****************************************************************
> > >
> > _______________________________________________
> > freebsd-questions at freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> > To unsubscribe, send any mail to "
> freebsd-questions-unsubscribe at freebsd.org"
> >
> >
> >
> > *
> >
> >
> >
> >
> >
> >
> >
> ************************************************************************************
> > This footnote confirms that this email message has been scanned by
> > PineApp Mail-SeCure for the presence of malicious code, vandals &
> computer viruses.
> >
> ************************************************************************************
> >
> >
>
>
> --
> Bill Moran
> Collaborative Fusion Inc.
>
> ****************************************************************
> IMPORTANT: This message contains confidential information and is
> intended only for the individual named. If the reader of this
> message is not an intended recipient (or the individual
> responsible for the delivery of this message to an intended
> recipient), please be advised that any re-use, dissemination,
> distribution or copying of this message is prohibited. Please
> notify the sender immediately by e-mail if you have received
> this e-mail by mistake and delete this e-mail from your system.
> E-mail transmission cannot be guaranteed to be secure or
> error-free as information could be intercepted, corrupted, lost,
> destroyed, arrive late or incomplete, or contain viruses. The
> sender therefore does not accept liability for any errors or
> omissions in the contents of this message, which arise as a
> result of e-mail transmission.
> ****************************************************************
>