Building a mail application.. some advice appreciated

[David Southwell]

On Wednesday 25 April 2007 12:12:44 Jeffrey Goldberg wrote:


> > Hi
> >
> > I am looking for some general advice and guidance for selecting
> > software
> > components to fulfill a proof of concept test.
> >
> > I need a mail application with  features requiring that incoming
> > mails, which
> > should comply with a predetermined format, be initially examined for
> > compliance with that format
>
> Presumably you are talking about the the format of the body of the
> message.  
Thanks for getting back to me

subject and body needed to be checked for data content (which is in the form 
of command strings) and format validity plus  data content checking for a 
number of headers.
> Or will the formated information all be in the headers? 
> The tools you use will depend on that (and other things).
>
> > Each sender (read user) has to be uniquely identified in the
> > database system
> > and a log kept of every mail received.
> >
> > Sender verification requirements are high and, among other things,
> > the output
> > from attachment processing must provide an input to the
> > verification system.
>
> In order to help understand what you need it is important to break up
> the word "verification"  and by "sender".  In mail transport jargon
> "sender" means the address passed as the argument (an email address)
> to the MAIL FROM directive during SMTP.  Do you mean that or do you
> mean something else by sender.
>
> By verification, do you just want to know whether it is a valid
> sender or do you want to authenticate the sender?  


Both. 
> That is, do you 
> want to have confidence that the sender is who they say they are?
>
> When dealing with things like "verification" people like to break
> down the muddle into distinct tasks.  One is "identification".  For
> example, a username usually works to *identify* a particular user, so
> we know what user we are talking about.  "Authentication" is how we
> know that the person (or entity) wanting to use that identify really
> is who they say that are.  For example knowing a password is used to
> authenticate.  (Then there is "authorization") which I will leave
> aside for now).

Iddentification and authentication processess are assisted by the email 
containing attachments which are small one time use files that should only be 
in the possession of a specific valid user.
>
> For email there already are two good systems for authentication.  PGP
> and S/MIME.  Most toolkits for dealing with mail usually come with
> easy support for either of those.  For example, along with the many
> perl modules that exist for dealing with mail, there are modules for
> processing PGP or S/MIME signed messages.
>
> > Mails that pass verification requirements are to be initially
> > processed by the
> > receiving server and the results of verification transferred to a
> > mysql
> > database.
>
> Again, that depends on what the processing actually will be.  You
> will not do this within the MTA, but will pass (usually through an
> alias to a pipe, just like in mailman) the message to some program or
> script.  Whether it's written in perl, python, awk  etc is up to you,
> though there will already be nice packages in perl and python (and
> lots of other choices) for doing this kind of thing.  You will want a
> system that has nice integration with MySQL.  Again, all of the
> popular scripting languages do.

I am familiar with gawl/awk and php.

>
> > Attachments  have to be extracted and passed for processing and
> > results
> > stored  in a mysql db.
>
> Again, you will want to use some system that has modules or libraries
> for dealing with email attachments.  Perl is what I'm most familiar
> with, but all the other ones will have such libraries/modules as well.
>
> > Mails that do not comply with the verification requirements need to
> > be passed
> > to another server for logging and processing.
>
> When you say another "server" do you mean some other service to deal
> with these, some other mail server or some other host?
another mail server
>
> > The system has to be scaleable.
> >
> > I realise I have not given a lot of detailed information but here
> > is the rub I
> > need to build, as quickly as possible, a proof of concept, using
> > readily
> > available software components.
>
> In a sense, any mailing list management system (that can use a mysql
> backend) already does what specify.  So do many other things that
> process mail (like customer relations systems, or bug/ticket
> reporting systems).
>
> > The OS is freebsd (currently 6.1) running postfix.
> >
> > What components would you choose for this exercise? Be able to
> > build quickly
> > and easily is the priority  for this stage and low server demand
> > would not be
> > some important at this time.
>
> I would really need to know what the problem is that you are trying
> to solve really is.  

The focus of the application is around the transmission of small one time use 
encrypted files by identified and authenticated users attached to the emails 
plus the interpretation of terse formated data found in the subject and body 
of the email.

Email is the only practical solution. 
> That way, we'll have a better understanding of 
> whether email is the right solution, whether this has already been
> done, what kind of authentication really is needed, etc.  


> So tell us 
> what you are trying to do with the system, and it will be easier to
> make meaningful recommendations.
>