Spam Filter Efficiency

[Lyle Evans]

At 11:58 PM 11/21/2007, you wrote:
>Greetings List,
>
>I apologize if this topic has already been raised, however I would 
>like some feedback on how people are managing spam in high volume 
>email environments.
>
>To give you a little background, our organization currently has 
>three reasonably powerful boxes (dual XEON with 4GB ram), processing 
>about 800000 messages
>a day (total) and are seriously struggling under the load.
>
>Our configuration consists of Postfix with a couple of RBL checks, 
>GLD greylisting (central MySQL db), which falls through to 
>MailScanner which filters
>with SpamAssassin / Clamav.
>
> >From the mail scanners, the emails are the forwarded (via an LDAP 
> lookup) to a specific Cyrus mail store.
>
>We have turned off DCC checks in SpamAssassin which has improved 
>performance quite a bit, however we are still doing Razor checks.
>
>We have investigated a couple of commercial solutions which clamed 
>to be able to handle more than our quantity of mail on one box, 
>however the spectacular
>pricetags associated with such solutions suggest we won't be moving 
>forward with these any time soon.
>
>We are also looking at other open source solutions such as amavis / 
>dspam to see if we can try and improve the throughput on our current hardware.
>
>What I would like some feedback on is if anyone has already gone 
>down this path and found one solution that performs better than 
>another, or if anyone is
>using a similar setup to ours and has found better ways to optimise it.
>
>I would very much appreciate some feedback either on or off list 
>please, as to how other people might be tackling this same problem.

You are perhaps not on the best list for asking this question, not 
that its off topic here
just you might get a lot more useful responses the Spamassassin 
and/or Mailscanner lists.

You need to give a lot more information in order for people to help you.
To start with the software versions you are using. For example there 
have been a number of
performance improvements in Spamassassin 3.2.x that may make a great 
deal of difference.
(short circuiting of definite ham and spam, and sa-compile that 
allows rules to be compiled).
How many Mailscanner processes are running at single time? Is your 
machine thrashing i. e.
hitting the swap? (If you are swapping try lowering Mailscanner's 
Maxchildren until it stops swapping.
You probably want it just below the swapping point. )
Do you have a local caching DNS server on each Mailscanner box?
If you are  bottle necked on CPU and not swapping then you need to 
take a careful look
at your SA rule sets and make sure you don't have excessively large 
ones, try sa-compile in
3.2.x and short circuiting etc. Also try to determine what is using CPU.
(Make sure your are up to date with sa-update.) If you are
bottle necked on I/O on a disk then ...   etc.

In other words you probably need to give considerable performance 
statistics etc. before people
can you give useful suggestions. Its quite possible that significant 
performance enhancements
can be made in mailscanner and/or Spamassassin with careful tuning.

Regards,
Lyle Evans
Blacksburg.Net