[thelist] Serious antispam measures

Kelly Hallman khallman at ultrafancy.com
Mon Apr 19 06:47:11 CDT 2004


Apr 19 at 10:48am, Kasimir K wrote:
>  > Sounds tiring for the scant quantity of legit email I get. And what
>  > about my poor mother? I have found the subject line route to work
>  > pretty well, and it didn't have to be that complex.
> 
> Trying to find logically solid ways to this utterly fascinating problem:

I have a pretty effective spam filtering system, it's rare that a spam 
finds it's way through. It's easy to manage the few that do.

1. I sign up with junkmail at mydomain.name anywhere that I don't expect
   will be mailing me anything important. I have a filter in my email
   client that colors all emails to this address in red. By coloring
   messages this way, scanning my inbox is very quick. However, I can
   also catch registration emails and other things sent to my junkmail
   address, when I'm expecting them.

  [Since an email catch-all is a prerequisite, I used to use
   yourcmpnyname at mydomain.name but eventually concluded even if I could
   nail down who was selling or giving my address, there was little 
   recourse or incentive to do it. I don't know any stats, but 
   potentially doing that could also get you on spam lists with many
   different addresses, and increase your spam exposure.]

2. I run a Bayesian mail filter. This is by far the most effective type
   of spam filter out there, as it trains itself based on what you tag
   as spam or legitimate email (I trained it on about 10000 spams tagged
   by SpamAssassin, which I used to use, and about 5000 legitimate mails).
   A filter in my mail client moves mails tagged by BMF to a spam folder
   after they are tagged as spam by BMF.

3. My BMF tends to be more prone to false-positives than false-negatives,
   (understandable in most cases, for instance most mail I get from
   aol.com is spam, so when someone on AOL emails me, the filter 
   logically considers it spam. To compensate for this, I implemented a 
   whitelist with procmail. The combination of a whitelist and a BMF is
   very effective. I scan my spam folder about once a day and whitelist
   any legit emailers that end up there. Since most of the spams are to
   my junkmail address anyway, it's very easy to see the messages that
   are likely not spam, since they aren't colored red.

4. Though the above measures are enough, I also implemented a blacklist
   and a banned subjects list, since some spams seem to be highly
   repetitive, either from the same sender/domain, or same subjects.
   The messages are usually filed properly, but I know there are
   some emails that I can safely dump sight unseen.

To further streamline my email operation, I also have color rules for
mailing lists and other things that tend to pile up in my inbox.

I don't believe in any solution that makes life harder for legitimate
emailers. When I get an email back from someone that offers a weak excuse
about their need to control spam, and says that I need to take some
particular action, I just consider that person lame. I get more email--and
more spam--than most people I know, and I am able to control it. :)


References and Additional Information:

I use http://sourceforge.net/projects/bmf/ though I know that there are 
even better implementations of Bayesian filters out there. If you are 
unfamilliar with Bayesian mail filters, I suggest this visionary article
on the topic: http://www.paulgraham.com/spam.html -- a great read!

A whitelist or similar can be implemented in .procmailrc like so:

:0
* ? formail -zxFrom: | grep -i -f ${HOME}/.whitelist
| formail -A"X-Sorted: Whitelist" >>$DEFAULT

Then, add a filter to key off the X-Sorted field--or in my case, this 
recipe is processed before the recipe that filters the mail through bmf, 
so if the user is on the whitelist, the email is never spam-checked.

The ~/.whitelist file is simply a list of email addresses/patterns. To
implement a blacklist, change the filename to grep and the delivery rule
(the last line in a procmail recipe) to /dev/null. To make a list of
banned subjects (or any other header), change the field formail checks.

I'd love to think there was a way to reduce the amount of spam that is 
sent, but with a filtering regime this effective, I say, bring it on! 
Since I know they will anyway... procmail is your friend!

-- 
Kelly Hallman
// Ultrafancy



More information about the thelist mailing list