SpamAssassian with FreeBSD and Big Mail Server

[Vahric MUHTARYAN]

Hi Oliver , 

	We are using SA-exim, and we are not scan for viruses just only for
spam but I don't agree with you about using memory, because on our test
environment I saw that we did not use too much RAM . I wonder We are reaced
limits or not :) ... I will test with your given commands what happened. 

Thanks for your sharing knowledge and help Oliver
Vahric 
  

-----Original Message-----
From: owner-freebsd-isp at freebsd.org [mailto:owner-freebsd-isp at freebsd.org]
On Behalf Of Oliver Brandmueller
Sent: Wednesday, February 23, 2005 2:24 PM
To: freebsd-isp at freebsd.org
Subject: Re: SpamAssassian with FreeBSD and Big Mail Server

Hi.

On Wed, Feb 23, 2005 at 01:01:14PM +0200, Vahric MUHTARYAN wrote:
>             Really I don't know can I say a big mail server which have
> 30,000 mailbox on it 1200+ simultaneously connections (pop,smtp,webmail).
> ?ncoming smtp connections are between 200-400 . We want to run spam
software
> on it but machine can't handle it for this reason we seperated machine
> freebsd+exim+SpamAssassian but on 400 connection machine goes down average
> is very high , cpu usage really too high . 
>  
>             I want to learn Anybody Who have closer or bigger system and
> using SpamAssassian ?! 
> Really this 400 connection simultaneously can be limit for spam software
?!
> I mean Anybody can handle more ?! 
> I have to design distributed environment ?! 
>  
> My Hardware is (for spam)
>             2 X PIII 1G +  1 GB RAM + 2 DISK RAID 0 SCSI 10000 RPM   

You don't tell us, how you use SpamAssassin currently.

If you use spamd for connection time checking by exiscan keep in mind, 
that each mail connection lasts longer, so you need a lot more 
filedescriptors than before on the machine with the MTA!

I'm currently using amavisd-new (want to go away from that, it's eating
too much resources) on 4 loadbalanced Dual Xeon machines. We scan for
spam and viruses. Each of the machines is able to filter about 400.000
mails per day, given that you have peak times and ow traffic times. 
(RAID 10 over 4 10krpm disks on hardware RAID, dual Xeon 2.4 GHz, 2 GB
RAM/each machine). spamd itelf won't need so many resources as
amavisd-new (this is a legacy setup).

SpamAssassin needs a lot of RAM. Also keep in mind, that usually you are 
asking different nameservers for blacklists and stuff like that. That 
means, that filtering an e-mail takes a certain time, no matter how fast 
your system is. Dual P3 1 GHz and nly 1 Gig of RAM looks pretty much 
like a bottleneck to me, even without all the amavisd overhead.

Have a look at "top", at "systat -vmstat", "iostat -d 1" and such, to 
get a clue, if your processors are too slow (only little I/O, enough 
free memory, 0 idle time over long periods), if you are lacking RAM (few 
free memory, lot's of I/O) or if you have an I/O problem (a lot of 
processor idle time, but the disks having >>400 ios per second all the 
time).

My Spam filters are not quite the same as your, as the still have their 
MTA, do virus checking and have queues, so they need a lot of IO 
anyways. The don't have swap space: as soon as they start swapping, 
things get worse, becuse there's too much disk io and they get very 
slow. So have a look at your swap space. Are you using Swap at all? When 
th machines get loaded, do they start swapping very hard? That'd 
probably kill them.

Another thing: using RAID 0 is VERY RISKY in any mail environment. RAID 
(especially in this case) is not so much about data security (if the 
spamd breaks the connection, the mail is still not lost), but about 
availability. You are close to the edge in your environment. one disk 
breaking would mean hours (if not days!) without accepting mails or 
without filtering. I would really think that over.


- Oliver


-- 
| Oliver Brandmueller | Offenbacher Str. 1  | Germany       D-14197 Berlin |
| Fon +49-172-3130856 | Fax +49-172-3145027 | WWW:   http://the.addict.de/ |
|               Ich bin das Internet. Sowahr ich Gott helfe.               |
| Eine gewerbliche Nutzung aller enthaltenen Adressen ist nicht gestattet! |
_______________________________________________
freebsd-isp at freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-isp
To unsubscribe, send any mail to "freebsd-isp-unsubscribe at freebsd.org"