OT - Scalable email server solution needed
wmoran at collaborativefusion.com
Sat Apr 8 11:40:37 UTC 2006
Duane Whitty <duane at greenmeadow.ca> wrote:
> My appologies in advance for the OT post.
> This probably more appropriately belongs on ISP@ but
> there doesn't seem to be much traffic there.
> To the point: I'm about to provide hosting services,
> including but not limited to email, to a few local
> business. Right now I have Sendmail configured
> and I was about to install Courier-IMAP. My
> concern is future scalability. I'm not sure sendmail
> is the right back-end MTA. Right now Sendmail
> and Procmail will work fine I believe with the addition
> of an IMAP server. But what about later on?
> I was looking at DBmail but I'm not sure just how
> much activity that project has so I'm a little leary,
> but perhaps that's unjustified? So now
> I'm considering Exim or POSFIX. The Sun Java
> Communications Server looks appealing but unless
> I misunderstood I don't have anywhere near the
> bankroll for that yet and it doesn't look like a good
> fit for a start-up operation anyhow.
> I like that POPular port and the way it uses proxies etc.,
> but it doesn't seem to support IMAP.
> Is there anyone on the list who might be serving many
> business customers each with many users who would
> be willing to share their insights or opinions on this?
> From my past experiences, transitioning from a low volume
> solution to a high volume solution, regardless of the application
> domain, is usually best solved by making the right choice in
> the beginning. It would seem like an important marketing
> tool as well.
There are a number of excellent tools available, I will give a few
opinions on what I like best.
I think the most important key, however, is that your data be in a
format that's flexible enough to scale, while also giving you the
ability to choose tools. This being said, standardize on Maildir as
your backend storage system right from the start.
Maildir has a number of advantages over mbox. A few quick links:
One advantage of Maildir over mbox and the Cyrus db (that, for some reason,
I seldom see touted) is that you can make a safe backup of a Maildir without
shutting down the SMTP, POP, or IMAP server. With both mbox and the Cyrus
mail system, you have to shut the mail system down to back up the mail
boxes. You may or may not have a need to make mail box backups at this time,
but it'll be nice to be able to do it if a customer is willing to pay for
it! With both mbox and Cyrus, if you back up without stopping the server,
and entire mailbox will be corrupted if the file holding it's mail is in
use during backup and restoring will be difficult or impossible. With
Maildir, only the one email that was in use will be corrupted on the
Once you've chosen to use Maildir, you can choose which softwares you want
to use to get mail into and out of your Maildirs. Almost all Unix mail
software supports both mbox and Maildir, so you have a LOT of choices.
Additionaly, if you find that you picked a poor IMAP server for your
needs, you can easily switch later without needing to convert all your
data. You can even run multiple different IMAP and SMTP systems to try
them out - on the same Maildirs.
I recommend you set up an SQL database to keep your users in. This will
result in a bit more work up front, but it scales much better. You'll be
able to use tools like PHP to create easy to use scripts to manage your
email users, and you don't have to spam your /etc/password file with
hundreds (or thousands) of users that shouldn't be allowed to log in
anyway. In my experience, this gets forgotten, and user accidentally end
up with a login shell - since the admins aren't securing the mail server
against local attacks, these accidental login shells become a security
Now, once you've decided to use Maildirs with an SQL user database, it's
time to pick software.
I recommend PostgreSQL for the DB. It has all the performance of MySQL,
but it also has a lot of advanced features that MySQL doesn't that will
allow you to scale up, such as mature transaction support, point in time
recovery of databases and mature stored procedures.
For the SMTP system I recommend Postfix. It has all the security advantages
of qmail, with an easier to understand configuration and a lot more commonly
used features built in. It supports Maildir and PostgreSQL databases.
For the POP/IMAP system, I recommend Dovecot. I've been using it since it
was beta and it just works. The config is fairly straight forward. It
supports Maildir and PostgreSQL databases, and is written with a careful
focus on security and performance. In my experience it works well with
all MUAs, even those from Microsoft. I've never found it lacking for
features, and I've noticed that my IMAP server is cleaner to use than a
lot of others I've seen.
As I said before, however: if you go with a good backend storage format
(Maildir) you'll find that if you don't like Postfix or Dovecot, it won't
be a _huge_ chore to switch to something else. Since Maildir is reliable
over NFS, you can even run multiple computers all off the same backend.
(Although newer NFS implementations make this less of a concern than when
Maildir was first developed)
PostgreSQL is a little more of a commitment, but it seems as if support
for PostgreSQL is growing, and has been for many years. So I don't think
it's a dangerous bet.
More information about the freebsd-questions