Mail.app spam filter a dud?

by Charles Miller on April 6, 2003

Note: Since I was linked to from OSOpinion, I should add a pointer to the follow-up, where I found the problem, and it was at least partly my fault. Mail.app now catches 90% of my spam.

One of the big selling-points for Apple's Mail application in OS X is the adaptive spam filter. Reviewers have gone wild praising how wonderful it is, and how it gets rid of 95% of their spam.

I used to think this too. It seemed to work really well on my Powerbook, but when I changed to my iMac, despite an enormous amount of training, it seems to have lost the ability to properly decide what is, or is not spam. From observation, I decided that it probably catches about a fifth of the junk, the rest ends up in my Inbox.

So I performed a test. Rather than deleting my junk-mail, I've been storing it in a folder. Today, I stripped out the header-lines proclaiming the mail as junk, and ran it back through the junk-mail filter. Note, everything in this mailbox had previously been marked as spam, and had thus contributed to whatever database the filter uses to choose which mail to flag.

Out of 1080 of these spam emails, the filter caught 268. The remaining 812 did not register as junk. I take this to mean that either:

  1. Apple's adaptive spam-filtering algorithm is ineffectual.
  2. Spammers have worked out how to evade the filter
  3. I have an incredibly atypical mail profile.
  4. I somehow screwed up the filter with bogus data or misconfiguration.

Regardless, the difference between a filter that only catches a fifth of the spam and no filter at all is barely noticeable. It means I still have to go through and manually delete large scads of mail based on their subject and senders once or twice a day.

How is everyone else faring? I suspect my first guess is correct, and the filter is just too lenient. I can't rule out the other three options, though, without more data. (Note, this doen't mean “suggest some product that works better”, I can do that research for myself. I'm only curious about Apple's implementation right now)

Previously: A nerd in a fishbowl

Next: Brain Stew