Non English Spam

Ted Mittelstaedt tedm at
Fri Oct 20 00:29:43 PDT 2006

----- Original Message ----- 
From: "Erik Norgaard" <norgaard at>
To: "Ted Mittelstaedt" <tedm at>
Cc: "Beech Rintoul" <freebsd at>;
<freebsd-questions at>
Sent: Tuesday, October 17, 2006 2:22 AM
Subject: Re: Non English Spam

> Also this means that later filtering on the first Received field is
> double work: You already accepted the mail based on that information.
> In short: Writing header filtering rules for the Received field is
> simply waste of time and proof of inefficiency.

I agree with this but unfortunately the real world often screws this up.

For example, SpamCop is one of the most effective blacklists on the
Internet because of it's high user participation.  Unfortunately, it
repeatedly blocks yahoomail, craigslist, and ebay because spammers
hate it and try to stuff it up so as to get people to stop using it.

As a result, you cannot use Spamcop to reject at the HELO.  Instead
you have to post-filter the mail and do your spamcop lookups, so
you can exempt domains like ebay that are legitimate.

> Just as Sendmail, Postfix is not designed for spam filtering. Postfix
> provides simple filtering mechanisms, keeping it simple postfix provides
> an effective and reliable MTA that doesn't suffer the track record of
> security bugs Sendmail does.
> When the native filters does not suffice you can combine with any number
> of "policy services": External filtering mechanisms such as postgrey,
> spam assassin etc. This design is clean, reliable and easy to manage.

Same for Sendmail, you can use milters to add all this stuff in.  Or you
can do it in the local delivery agent.

> OP requested a way to filter away the spam in foreign character sets
> because for some reason these were not caught by Spam Assassin or
> procmail. I gave a solution that solves that problem, and I mentioned
> the problem of false negatives for this list.
> Rather than get pissed, do try to offer an alternative solution to a
> real problem.

There really is no solution.  Fundamentally, well written spam is
not distinguishable from non-spam by a computer.  What has saved our asses
far is that there's not a spammer alive who has been able to resist the
to use bold, colors, blinking test, hot phrases, and other attention-getting
devices in their spams.  Since you can program a computer to look for the
attention getting stuff, what has happened is a little social engineering.

Most people today have abandonded use of attention-getting devices
in their e-mails because when they use HTMLized text and such,
their mails tend to get blocked as spam by everyone and their dog.
So, the spam content filters can still distinguish spam from non-spam
by looking for these differences.

But it is only a matter of time before the spammers all wake up and
smell the coffee, and start using standard ASCII pure text for their
spams, and then all these charset filters your loving will go gurgling
down the drain.

Granted that might make their spams less effective so they might
get less respondents.

Frankly, I think there is no technical solution, I think there are only
political solutions.  We've already made spam illegal in the US, and
the CAN-SPAM act defines the "advertised" party in the spams
also as a spammer, in addition to the actual spammer sending the

It would be childs play for the FBI to work with the major ISP's
to create thousands of dummy e-mail addresses and use these
to capture spam runs.  Then they just go arrest the people in the
company that is being advertised and hang a few of them high.
There's no need to even go after the actual spammers themselves.

When this happens enough times, the supply of companies that
are willing to pay spammers to send spam will dry up, and the
spammers will go find some other criminal activity to engage in.

But, the FBI isn't doing this because many of the companies that
are hiring spammers have lots of money, and that gives them lots of
political power.  So, the will to curb spam just isn' t there even though
money "earned" by spammers is undoubtedly going into organized
crime, feeding terror cells, and other more nasty stuff.

> I asked politely if there were any consensus or best practices etc. on
> this issue. You have the regular mail on "how to get the best results"
> there are recommendations on how to use this list, they are not enforced
> but only serve as guidelines.
> I don't try to force people to use particular character sets, I merely
> ask whether such recommendation exist for "the best results when using
> the list", in which case filtering on charsets may be the least
> imperfect solution (until you share your perfect filter, that is).

Your continuing to try to muddy the issue by inferring that personal
filters are the same as requirements to post.

You snipped all my explanation of what the differences are and responded
with a snotty request for a perfect filter, when I never said I ever had

As I already stated, what people do on their own mailserver is their
business.  If they want to filter Asian charsets, then fine.  Go ahead.
But, telling people they can't use them when posting to the list is
crossing the line.

Certainly a "best results when using the list" document is a good thing.
But, that is a recommendation, not a requirement.  The response that
got me pissed was speculating that the list server should filter on Asian
and we should order, not recommend, to
people that they don't use Asian charsets.  I'm glad to see your
backwatering from that.

The charset filtering may well be a "least imperfect" filter, but it
must be implemented on the recipients of the list mailservers and
mail clients.


More information about the freebsd-questions mailing list