Running large DB's on FreeBSD

[Jim C. Nasby]

On Fri, Oct 27, 2006 at 10:06:30AM -0400, Bucky Jordan wrote:
> > -----Original Message-----
> > From: owner-freebsd-stable at freebsd.org [mailto:owner-freebsd-
> > stable at freebsd.org] On Behalf Of Jim C. Nasby
> > Sent: Friday, October 27, 2006 12:44 AM
> > To: David Magda
> > Cc: Mike Jakubik; stable at freebsd.org
> > Subject: Re: Running large DB's on FreeBSD
> > 
> > On Mon, Oct 23, 2006 at 08:15:04PM -0400, David Magda wrote:
> > > As for Postgres on FreeBSD, FlighAware seems to be using it some
> some
> > > decent amount of data:
> > >
> > > >. Receiving the data and processing it puts them about 6 minutes
> > > >behind real time
> > > >. Generating one map can be done in about 160 milliseconds of CPU
> time
> > > >. Capable of generating several million maps a day
> > > >. About 1 TB of stored data
> > > >. Approximately 40 million position updates on air craft per day
> > >
> > > http://joseph.randomnetworks.com/archives/2006/05/12/flightaware-
> > > freebsd-and-postgresql/
> > 
> > And that's on a dual opteron with 12G of memory and a run of the mill
> > RAID10 (for the database that is).
> 
> Yes.. but how many disks (size/type/rpm?) are in that RAID 10? I'm
> guessing it's an external enclosure... 
> 
> Also, I know 10k rpm vs 15 doesn't make much of a difference for
> sequential, but random IO seems to be significantly improved. Granted,
> it's not as dramatic as adding more spindles...
 
IIRC it's a 6 drive array of SATA. Nothing all that fancy.

> I think the other point that may be relevant is the active section of
> the data that you're accessing, and how good your design is in terms of
> being able to access that directly. You could have a 1TB database, but
> only have a portion that is frequently accessed/updated. In that case,
> you might just need lots of storage, which is fairly inexpensive these
> days. Also, your money might be better spent on more RAM- if you can fit
> most of the active data in memory, that will also have a positive impact
> on performance.
> 
> As pointed out, 10GB isn't really that much, especially when you can buy
> relatively inexpensive servers with 8 or 16 GB of ram. Fitting over half
> your db in memory is quit a luxury.

Well, what's most important is your system architecture. If you have a
poor design to start with, you'll never get good performance out of it.
-- 
Jim C. Nasby, Database Architect                decibel at decibel.org 
Give your computer some brain candy! www.distributed.net Team #1828

Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"