Non English Spam

[Ian Smith]

On Sun, 15 Oct 2006 freebsd-questions-request at freebsd.org wrote:
 > Message: 2
 > Date: Sun, 15 Oct 2006 12:47:37 +0200
 > From: Erik Norgaard <norgaard at locolomo.org>

 > Ted Mittelstaedt wrote:
 > 
 > >> I have noted however, that some subscribers to this list write english
 > >> encoded in one of the above character sets, I don't know enough about
 > >> the character set definition, but it seems that English characters are a
 > >> subset of any character set?
 > >>
 > >> What is the recommended policy here? Should subscribers be advised to
 > >> change character set when posting to the list?
 > > 
 > > No.  It's the responsibility of the person doing the filtering - in this
 > > case you -
 > > to exempt any known good e-mail sender from your filters.
 > 
 > > You know damn well that legitimate mailing list mail comes from
 > > 
 > > mx2.freebsd.org (mx2.freebsd.org [216.136.204.119])
 > > 
 > > it's right in the headers of the messages on the list.
 > 
 > First: You know all too well that filtering based on "Received" header 
 > fields is not reliable - any decent spammer know how to forge that. 
 > Accepting mail from a particular host should be done even before the 
 > mail delivery starts.

Ted's talking about the _first_ Received header, see mine below.  It's
the only one you _can_ rely on, assuming your mailserver isn't lying to
you.  Subsequent headers, sure, all can be faked, trust noone .. :)

 > Received: from mx2.freebsd.org (mx2.freebsd.org [216.136.204.119])
 > 	by gaia.nimnet.asn.au (8.8.8/8.8.8R1.4) with ESMTP id WAA18000
 > 	for <smithi at nimnet.asn.au>; Sun, 15 Oct 2006 22:02:19 +1000 (EST)
 > 	(envelope-from owner-freebsd-questions at freebsd.org)

There's the verified IP address of the connecting peer mailserver, that
IP's reverse resolution from DNS, and the HELO presented.  Any and all
of which can be analysed, looked up in maps, blacklisted, whitelisted,
or filtered any way you want, no? 

 > Second: If you know postfix, you also know that header filtering is 
 > independent of other checks, even the result of filtering on individual 
 > header lines are independent.

Does that mean you can't black/grey/whitelist by connecting mailserver?

 > So the ideal you mention is not an option until a complete public list 
 > of authorized mail servers is available and all mail relayed through 
 > these requires authentication.

That's the 'solution' the mega players appear to be proposing.  And who
then authorises whom to run mailservers?  What about, er, us?  Shudder. 

 > Or do you have the solution that does not imply accepting any of a 
 > myriad of character sets?
 > 
 > I'd be happy to implement that, but I don't want to open my mail server 
 > to receive mail I have no means of reading and understanding just 
 > because it is RFC compliant.

Like any one, you can reject any mail you don't fancy, for whatever
reason you don't want it.  That doesn't require proposing that others
should do likewise, as in wanting to specify 'standards' for lists.

As Ted pointed out, various people often post perfectly intelligible
messages in English in the various FreeBSD lists, reporting non-Roman
charsets.  I could mention one regular poster (and committer) whose
messages provide no charset information at all :)

 > > You have no right to
 > > force other people to conform to what you feel is acceptable formatting
 > > of their message as long as they meet the SMTP rfc standards.  That's
 > > why we have RFC's.
 > 
 > You you know perfectly well that content filtering is not based on the 
 > RFC's on SMTP but rather on the Internet Message Format and various 
 > RFC's on MIME - but I assume that you meant to refer to these.
 > 
 > Basically what you say here is that spammers have every right to flood 
 > mail servers as long as they do so compliant with the RFC's?

Have you noticed a lot of non-Roman charset spam on the FreeBSD lists?

 > I don't force anyone to conform to any arbitrary standards that I decide 
 > upon, but I have every legitimate right to reject anything that doesn't 
 > conform to my arbitrary standards.

Of course.

 > Yet, it is somewhat implicit that this is an English language list, any 
 > one writing in a different language may be lucky to find someone who can 
 > respond in their language, but are just as often referred to one of the 
 > language specific lists - if their message is not simply ignored.

We're not - with respect to suggesting 'rules' for these lists - talking
about non English language messages.  As you say, they get dealt with,
often offlist, by someone helpful who knows that language.  So this is
about whether to 'enforce' particular charsets for messages in English.

 > So we do actually impose some arbitrary rule on subscribers, namely to 
 > write in English. Given that we find it reasonable to impose such a 
 > rule, then why is it unreasonable to impose that they should abstain 
 > from obscure non-English character sets?

Because it's unnecessary, as well as arbitary, to filter list messages
by charset alone as an unassociated variable.  Sure, it might be a hint
in the mix to give some points.  The FreeBSD lists are mostly incredibly
spam free, but I doubt that much of that filtering is based on charsets.

 > I was hoping to find a way that we can all get along, I find it kind of 
 > useless to waste my resources on mail written in languages that I have 
 > no means of interpreting.

So reject any mail you want to yourself, on whatever basis you choose,
so it's not a problem for you.  But please don't need everyone to do the
same.  You may miss some messages from some of those pesky Russians, for
example, but I often find them handy.

Cheers, Ian