e-mail server farm question

[Vulpes Velox]

On Tue, 23 May 2006 11:26:36 +0300
Evren Yurtesen <yurtesen at ispro.net.tr> wrote:

> Chad Leigh -- Shire.Net LLC wrote:
> 
> > 
> > On May 23, 2006, at 1:35 AM, Evren Yurtesen wrote:
> > 
> >> Vulpes Velox wrote:
> >>
> >>> On Mon, 22 May 2006 11:54:02 -0500
> >>> Kevin Kinsey <kdk at daleco.biz> wrote:
> >>>
> >>>>> At 07:17 AM 5/22/2006, Evren Yurtesen wrote:
> >>>>>
> >>>>>> Hello,
> >>>>>>
> >>>>>> I was wondering how does services like yahoo mail is storing
> >>>>>> e-mails. Somehow the smtp server should know where to deliver
> >>>>>> the mail inside the system and webmail should know from which
> >>>>>> server to read it from.
> >>>>>>
> >>>>>> Does anybody have any practical ideas about how it is done?
> >>>>>>
> >>>>
> >>>> Derek Ragona wrote:
> >>>> > If you are using sendmail, as most FreeBSD users are, you can
> >>>> > check the sendmail.org site for information on mail handling.
> >>>> > There are a number of methods that depend on your setup.
> >>>> >
> >>>>
> >>>> Well, it's pretty obvious that they aren't using a stock
> >>>> SendMail:
> >>>>
> >>>> # telnet mx2.mail.yahoo.com 25
> >>>> Trying 67.28.113.72...
> >>>> Connected to mx2.mail.yahoo.com.
> >>>> Escape character is '^]'.
> >>>> 220 mta309.mail.re4.yahoo.com ESMTP YSmtp service ready
> >>>>
> >>>> Short of finding an article written by someone 'in the know',
> >>>> or an answer for someone like that, we can only guess.  I'd
> >>>> probably start with guessing a big DB on a large SAN;
> >>>> which pretty much negates the "which server to read from"
> >>>> question (up to a point).  Everything else is pretty
> >>>> academic.  SMTP, IMAP, POP.
> >>>
> >>> Maildir makes it easy to distribute it across multiple machines
> >>> as well.
> >>
> >>
> >> What do you mean exactly? distributing 1 user's mails into
> >> seperate machines? I didnt understand how Maildir helps to this
> >> actually.
> >>
> > 
> > I am not sure anyone was talking about distributing 1 person's
> > mail across separate machines.  The discussion seemed to be how
> > to handle large amounts of mail spread out across machines, which
> > maildir helps with as you can have one or more file servers and
> > lots of consumers (imap/pop) and deliverers (mta) accessing those
> > maildirs on your file servers.  Combine with a backend database
> > of some sort (we use an  ldap db that includes the path for a
> > specific accounts mail) and voilá.
> > 
> > Chad
> > 
> 
> Ah sorry, I didnt think it that way for a moment. I thought you
> meant Maildir stores mails in seperate files compred to mbox format
> used by sendmail so...anyhow my mistake :) But it is possible to
> make changes to sendmail so that it will store to different folders
> also.

Maildir does store mail in serperate files. Each email is a file.
http://en.wikipedia.org/wiki/Maildir
 
> I think the conclusion is a database, multiple smtp servers
> querying database to see where to forward received e-mails,
> multiple pop3/imap servers querying database to see from where to
> read the e-mails and multiple storage machines. This way it can
> scale to an unlimited size.
> 
> So it requires a lot of coding :)

Nah, once you get everything installed and configured it is easy.
Dovecot and qmail are both easy to set up. Then just a bit of shell
scripting for a user adding and removing script.