huge email system

[CH]

Hello,

we run an outsourced mail relay service for some ISPs, with separate servers 
for outgoing and incoming mail.

we have two double-xeon 2.4 w/4G of RAM running FreeBSD-4.9, postfix-current 
with recipient address verification against different maildrop servers, and 
amavisd-new + spamassassin + antivirus for the incoming mail.

They both receive around 300mgs/min (which is around 430.000 msgs per server 
per day), of which 33% is being rejected, another 33% is being tagged as spam, 
1% is virus-infected, and the remaining 33% is being forwarded to the real 
accounts. both servers are running on the edge, with very little cpu available 
and the danger of swapping when too many outstanding amavis processes.  Use 
same hardware for same MX weight, and lots of memory for concurrent virus and 
spam checking.

for outoing mail we use several P4 2.4 w/1G RAM, in some cases we had to split 
the service into different machines due to different sasl settings, such as one 
with pam_radius and another one with rimap. they carry between 50 and 
200msgs/min, and go through amavisd too before being delivered.

The mail volume increase is impressive, we had to build a quota control into 
postfix to detect things such as too many bounces being returned to a spammer 
using one of our domains (this in the incoming mail relays) and a quota for 
outgoing mail (which isn't likely to happen if you have business accounts). 
These are all perl scripts that parse logfiles in almost realtime and update a 
central MySQL quota database: again, lots of CPU required.

We carry all this e-mail in and out for around 250k accounts of our customers. 
I know some of them use FreeBSD NFS boxes with 100M NICs on internal addresses 
without any problems. Backups and filesystem-snapshots are an issue though. 
Maildir makes backup/restore easier.

Encrypted passwords on SQL are a good idea if you need some sort of 
provisioning system for your accounts, then you might access them from radius, 
then from pam, and finally from imap and smtp+sasl among others.

Hope this helps,

-Carlos

Mensaje citado por Bill Vermillion <bv at wjv.com>:

> They all laughed on Fri, Nov 21, 2003 at 17:28  when David said:
>  
> > Hello -
> 
> > We need to build a stable, redundant, and speedy email system
> > that will last for a few years. We need to handle about 500,000
> > emails per day. We have about 30,000 users, so we need a lot of
> > storage.
> 
> 30,000 users with only 500,000 emails per day.  I say that based on
> running a small ISP with a few hundred users and see large mail
> volume.  All are business accounts.  
> 
> I'll let others comment on the rest, but I think 500,000 emails per
> day may be underestimating things.  That's only 20 emails per user
> per day.  
> 
> As to last a few years - who knows.  In the past year I've seen
> such an overall increase in mail volume that now I'm looking to get
> new servers with more CPU power.  It's not disk size that is the
> problem but the in-coming and out-going traffic that is killing the
> CPU.
> 
> Bill
> -- 
> Bill Vermillion - bv @ wjv . com
> _______________________________________________
> freebsd-isp at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-isp
> To unsubscribe, send any mail to "freebsd-isp-unsubscribe at freebsd.org"
> 
> -- 
> Este mensaje ha sido analizado en busca de virus y otros contenidos
> peligrosos,
> y se considera que está limpio.
> -- http://www.infodrive.com.ar - E-mail corporativo
> 
> 


-- 
Carlos

-------------------------------------------------
Mail enviado con INFODRIVE

----- Fin del mensaje reenviado -----


-- 
Carlos

-------------------------------------------------
Mail enviado con INFODRIVE


-- 
Este mensaje ha sido analizado en busca de virus y otros contenidos peligrosos,
y se considera que está limpio.
-- http://www.infodrive.com.ar - E-mail corporativo