Running large DB's on FreeBSD

[Bucky Jordan]

> -----Original Message-----
> From: owner-freebsd-stable at freebsd.org [mailto:owner-freebsd-
> stable at freebsd.org] On Behalf Of Jim C. Nasby
> Sent: Friday, October 27, 2006 12:44 AM
> To: David Magda
> Cc: Mike Jakubik; stable at freebsd.org
> Subject: Re: Running large DB's on FreeBSD
> 
> On Mon, Oct 23, 2006 at 08:15:04PM -0400, David Magda wrote:
> > As for Postgres on FreeBSD, FlighAware seems to be using it some
some
> > decent amount of data:
> >
> > >. Receiving the data and processing it puts them about 6 minutes
> > >behind real time
> > >. Generating one map can be done in about 160 milliseconds of CPU
time
> > >. Capable of generating several million maps a day
> > >. About 1 TB of stored data
> > >. Approximately 40 million position updates on air craft per day
> >
> > http://joseph.randomnetworks.com/archives/2006/05/12/flightaware-
> > freebsd-and-postgresql/
> 
> And that's on a dual opteron with 12G of memory and a run of the mill
> RAID10 (for the database that is).

Yes.. but how many disks (size/type/rpm?) are in that RAID 10? I'm
guessing it's an external enclosure... 

Also, I know 10k rpm vs 15 doesn't make much of a difference for
sequential, but random IO seems to be significantly improved. Granted,
it's not as dramatic as adding more spindles...

I think the other point that may be relevant is the active section of
the data that you're accessing, and how good your design is in terms of
being able to access that directly. You could have a 1TB database, but
only have a portion that is frequently accessed/updated. In that case,
you might just need lots of storage, which is fairly inexpensive these
days. Also, your money might be better spent on more RAM- if you can fit
most of the active data in memory, that will also have a positive impact
on performance.

As pointed out, 10GB isn't really that much, especially when you can buy
relatively inexpensive servers with 8 or 16 GB of ram. Fitting over half
your db in memory is quit a luxury.

- Bucky