Spam-Proof Your In-Box

Yes, you can turn the tide against junk e-mail. Our ratings of nine antispam tools reveal a surprising Best Buy.

Logan G. Harbaugh, a freelance writer based in Redding, California, tests spam filtering tools for InfoWorld and PC World.

Ask people what they find most irritating about the Internet, and you are almost guaranteed to hear a certain four-letter word: spam. In the past year alone, the volume of spam has ballooned--at some ISPs, junk messages now account for more than 75 percent of all e-mail. Fortunately, spam-filter software is improving.

As an analyst who reviews the kinds of filters used by large companies for InfoWorld (a sibling publication to PC World), I didn't expect desktop filters to beat enterprise-class products. After all, corporate spam services like Postini and Brightmail collect huge quantities of spam, and distill the essence of that junk to improve their filters. But two desktop tools I reviewed (including Cloudmark's SpamNet, our Best Buy) blocked more than 98 percent of junk mail--a huge improvement over the products in our May 2003 review, "Natural-Born Spam Killers." Keeping a low rate of false positives (real mail mistaken by the filter as spam) is critical, too. You don't want important e-mail to get dumped in the trash along with the herbal Viagra ads. SpamNet flagged only 17 of the 1082 nonjunk e-mail messages as spam.

I tested nine spam filtering tools in all (see " How We Tested Spam Filters" for the complete testing regime). Four filter products (from Network Associates, Panda, Symantec, and Trend Micro) are components of security suites reviewed in "Bigger Threats, Better Defense." In general, the suites with spam filters sorted through e-mail less accurately than did the stand-alone spam filter products--a result that jibes with the findings about antivirus and firewall software reported in that story.

For accuracy's sake, each tool required some initial "training" about kinds of e-mail it should not filter out. The four most accurate products in this review--SpamNet, InBoxer, IHateSpam, and SpamCatcher--did a good job with little training. These products collect information from users' filtering choices to help identify (and then create new filters to block) new forms of spam. When accuracy counts, human judgment trumps AI every time.

Aladdin Systems SpamCatcher

Of the filtering tools tested, SpamCatcher ($30) was among the simplest to install and operate. The tool integrates into Outlook, and will work with any POP3 e-mail client application. After training, SpamCatcher branded 46 legitimate messages as spam--a moderate amount.

A key component of the application is something Aladdin calls the SpamCatcher network. As you mark particular messages as spam, the program reports certain details about the junk e-mail to the network. The company then incorporates these details into software updates, improving the filtering accuracy for everyone.

All of the messages that SpamCatcher misidentified as spam were mass mailings, such as newsletters. Some filtering applications have a more difficult time with legitimate bulk messages because these often contain elements common to spam messages, such as unsubscribe information, prices, or links. The program didn't brand any nonspam messages from individuals as spam.

Audiotrieve InBoxer

InBoxer's controls are one mouse click away in the Outlook toolbar.

InBoxer ($28) beat down more spam than any other product in this review, while maintaining an acceptably low rate of false positives. InBoxer 1.1 works only with Outlook, from 2000 through 2003, and not with any other mail software. This limitation kept it from capturing our Best Buy award; but it's well worth a look if you use Outlook exclusively to read e-mail.

InBoxer's accuracy impressed me. The program let 26 out of 2135 spam messages--or about 1.3 percent--slip through. This level of accuracy puts InBoxer ahead of some sophisticated and expensive corporate filters.

The application mistook only one real message from an individual user for spam. It was slightly less accurate with legitimate bulk mail (such as e-mail newsletters I'd signed up for), but not to an extent that made the problem onerous to deal with. The program can filter spam out of any previously downloaded mail in your other folders, as well. Most of the other products we tested could only sort through new messages as they were downloaded.

InBoxer's integration with Outlook made training the filter simple. InBoxer creates two folders: 'InBoxer-blocked' and 'InBoxer-review'. Almost all false positives end up in the 'InBoxer-review' folder; since they're stored apart from the obvious spam, these messages are easy to sort through.

Cloudmark SpamNet

SpamNet keeps a running tally of how much time and money you're saving by using the application.

SpamNet ($4 per month) had the lowest false-positive rate of any spam filter I've ever tested (it didn't tag any legitimate messages that were not bulk mail) and a superb catch rate of over 98 percent. Its intuitive interface, virtually effortless filtering, and high success rate should put SpamNet on your short list of filtering products.

SpamNet integrates into the toolbar of Outlook 2000 through 2003 and of Outlook Express 5 and 6, and it can filter e-mail sent from any of these Outlook and Outlook Express versions. The program is free to download, but it costs $4 per month to use after a 30-day free-trial period ends.

Cloudmark collects data from every installed copy of SpamNet--more than 900,000 users in all. When you block spam manually, the program reports details about the addresses, URLs, subject lines, and text in the junk messages to the company, which uses this information to improve its filter accuracy. As a result, SpamNet sorted through junk mail extremely accurately, even before training ended. It missed just 37 spam messages (out of 2135 in the test pool), and recorded only 17 false positives out of 1082 legit messages--the fewest false positives of all the tested products.

Lyris Technologies MailShield

MailShield ($60) delivered the poorest filtering accuracy of the nine products I tested, permitting 796 spam messages--more than a third of the total volume of spam--to slither into the test system's in-box, even after training. Compounding the problem, MailShield incorrectly marked 141 of 1082 legitimate messages as spam.

MailShield can filter mail from any POP3 e-mail client. It runs as a separate program, idling in the system tray but springing to life when you check your e-mail. Through MailShield's interface you can delete spam messages before they get to your e-mail program, and you can pass along suspect but legitimate messages from their quarantine to your in-box.

Working through the separate interface makes training the software needlessly cumbersome. You have to launch the MailShield app, rather than using the familiar interface of your e-mail software, to identify spam or unblock false positives. And even with training, the application's accuracy at sorting out the junk left a lot to be desired.

Network Associates McAfee SpamKiller

SpamKiller ($35) is part of a security suite that also installs McAfee VirusScan. This can cause conflicts if you use another antivirus product on your computer. SpamKiller permitted 625 pieces of junk--nearly 30 percent of all spam--to pass through the filter, and it incorrectly identified 241 legitimate messages as spam.

The application shows the spam it has blocked--and lists the nonspam mail that it has passed along to the e-mail software--in its own interface. Managing messages through the SpamKiller interface took an annoyingly long time on my test system, which had a 1-GHz Pentium III processor and 512MB of RAM. To unblock legitimate mail, you have to click a single message, wait a few seconds for the program to acknowledge your selection, click the Rescue button, wait some more, and then move on to the next message. And because SpamKiller filtered out spam a lot less accurately than did most other products in this review, there were a lot of messages I needed to sort through--very slowly.

But the user interface's pokiness wasn't SpamKiller's only problem. There was no search function for finding all messages from a given sender, which would have simplified releasing batches of newsletter messages at once. And whitelisting doesn't release a sender's other e-mail from quarantine, either--you have to break each legit message out of spam-jail individually.

Representatives from Network Associates report that, after I completed my testing of SpamKiller for this story, the company released an updated version 5.1 of SpamKiller that addresses some of the system performance problems and user interface quirks that I noted during testing. Specifically, for SpamKiller 5.1, Network Associates increased the prominence of some of its more important features in the menu hierarchy, and made selecting and manipulating multiple messages easier than these tasks used to be. I was unable to test the update before press time.

Panda Software Platinum Internet Security

Panda adds a few user-friendly icons to the Outlook toolbar.

Panda's antispam product ($80), part of an Internet security suite, delivered by far the highest number of false positives (400 out of a possible 1082) of any of the programs I tested. Panda Platinum removed about 75 percent of actual spam messages, meaning that more than 500 spam messages ended up in my in-box after training.

Panda's suite installs an antivirus program, anti-spyware software, and a firewall in addition to the antispam tool. Unfortunately, it's all or nothing: You can't choose individual programs to install ? la carte, the installer gives you no option to refrain from installing other parts of the suite (you can disable them afterward, however), and you must remove any other antivirus program you have on your PC before Panda will install.

The Panda Platinum suite adds buttons to Outlook's toolbar to help you quickly whitelist false positives and identify missed spam for deletion. That's convenient, considering that I had to correct the more than 900 mistakes it made.

Sunbelt Software IHateSpam for Outlook

IHateSpam lets you blacklist a single address or an entire domain all at once.

IHateSpam 4 for Outlook ($20) delivered a high degree of filtering accuracy, with a very low false-positive rate. Only SpamNet and InBoxer caught more spam, but the difference was a couple dozen spam messages (out of 2135 total). Sunbelt sells versions customized to integrate with a particular program. I tested the Outlook version; other versions plug in to Outlook Express, MSN Hotmail, IncrediMail, and Eudora.

After locating and scanning the Outlook address book, IHateSpam added those addresses to its whitelist. The program adds a toolbar to Outlook that lets you cull spam from whatever mail folder you're browsing (a handy feature), designate messages as spam or legitimate, and whitelist or blacklist a sender's mail domain or address.

Among its unique features, IHateSpam can block e-mail that seems to be written in foreign alphabets--a common feature of spam that originates from Asia or eastern Europe. The application uses Sunbelt's Spam Learning Network Community to collect information about spam (such as the body and subject text) from users of the software; the data helps the company improve the filtering accuracy for all IHateSpam users. The 'Add to friends' menu item can exclude either specific senders or their mail domains from future blocking, while the 'Add to enemies' item blacklists individuals or domains.

IHateSpam slowed Outlook's responsiveness slightly, though not nearly as much as Norton AntiSpam did. With the program loaded on my test system, I detected a slight pause when I selected messages inside Outlook--less than a second, but noticeably longer than before I installed IHateSpam.

Symantec Norton AntiSpam

Norton AntiSpam 2004 ($40) stopped just under 80 percent of the spam in my tests, putting it solidly in the middle of the pack for accuracy. Its integration into Outlook, Outlook Express, and Eudora is a nice touch, but the program made my test system run dog-slow, though my hardware exceeded Symantec's minimum requirements.

After I installed AntiSpam, for example, clicking a mail folder to open it caused my PC to chug for several seconds before it reacted. Even when AntiSpam wasn't actively filtering, Outlook responded much more slowly when I tried to open messages, access folders, delete existing mail, or empty the trash. And downloading mail took about five times longer than it did before AntiSpam was installed.

Though the program can import the address book from Outlook, it took longer than 5 minutes to accomplish this task with our sample (which contained 1046 names and addresses). In contrast, IHateSpam took just 15 seconds to do the same thing.

Trend Micro PC-cillin Internet Security

Ratcheting up the aggressiveness of PC-cillin can create more false positives.

Like several other products from large, well-known companies, PC-cillin ($50) is a suite that includes a spam filter as part of a package with antivirus, a firewall, anti-spyware software, and parental controls. This product offered reasonable accuracy, catching 86 percent of my spam, but it was saddled with a fairly high false-positive rate of 6.9 percent.

But unlike any other product I tested, PC-cillin didn't actually delete anything. The program simply adds "spam:" to the beginning of the subject line for any messages it thinks are junk. You have to create a filtering rule in your mail software to deal with those messages, but that step isn't part of the installation.

Following the same pattern I saw with McAfee SpamKiller and Panda Platinum, PC-cillin doesn't play well with other applications. It checks for existing installations of antivirus products, and it requires you to uninstall any other antivirus application before you install the PC-cillin suite.

PC-cillin lets you adjust only one setting: a slider for establishing a low, medium, or high level of aggressiveness in identifying spam. And beyond adding a whitelist of e-mail addresses, you can't train the program to improve its filtering accuracy. Adding addresses to the whitelist isn't hard, but you can access the whitelist only through the PC-cillin interface, not from within Outlook. If you're considering buying PC-cillin's suite for its Best Buy-winning antivirus tool (see the Features Comparison chart), the spam filter is serviceable, but it's nowhere near the best of the bunch.

Features Comparison: Stand-Alone Filters Beat Suites (chart)

How We Tested Spam Filters

Freelance writer Logan Harbaugh tested most of the filters on Microsoft Outlook 2000; for the filtering software that didn't support Outlook, he used Outlook Express. Each product ran on a cleanly installed system with no software other than the operating system and Microsoft Office installed.

Depending on the product tested, Harbaugh used one of two test systems--one running the Windows 2000 Server operating system, and the other running Windows XP Professional. The hardware was a Blade Server--a slimmed-down rack-mounted computer with a 1-GHz Pentium III processor and 512MB of RAM. Some filters wouldn't install under Windows 2000 Server, so he tested those on the system running Windows XP.

Each product filtered the same 3217 messages, collected over a period of two weeks in March 2004, of which 2135 were spam and 1082 were legitimate messages.

Harbaugh trained the filters, when possible, with the first 1000 messages, adding senders to the whitelist (approved senders) or the blacklist (spammers). This was generally a two-step process: First, he sorted through e-mail that had been flagged as spam, and he identified messages that weren't really junk. Then he read the e-mail that had passed through the filter, and he identified spam that the filter had missed.

Typically, a filter offers a variable setting that determines how aggressively it filters mail. When dealing with a product that gives users a choice of settings, Harbaugh tested it at its default setting. Filter performance also depends on regular updates: When he had a choice, he left the frequency of updates at the default used by the program after installation; this ranged from once every hour to once a week.

The messages used for testing included several types of e-mail that are very difficult for most filters to diffrentiate from spam. Among these were press releases, legitimate bulk e-mail (both marketing and newsletters), mailing-list messages, and product updates from various companies. Much of the e-mail that the test account received came from senders not in the address book, which made things more difficult for some filters.

Challenge/Response: The 100 Percent Accurate Spam Filter

It has been several years since you could look for a specific word in e-mail to get rid of spam. Besides, to sneak past filters, spammers use different methods that change almost from hour to hour.

The most effective spam filters are permission-based tools, which block messages sent by anyone who isn't on your whitelist. That's great if your only wanted e-mail comes from friends and family members. But if you occasionally receive unexpected e-mail that you want to get, blocking every unknown sender won't work. That's where challenge/response, or CR, filtering apps come in. The term means that the filter will block e-mail from an unknown source unless the sender of the e-mail replies to a special message correctly.

Here's how it typically works: Right after your Uncle Sid sends you an e-mail, your CR tool sends him a message directing him to perform some action, such as clicking a link. If he responds correctly to the challenge, he gets added to the whitelist, and his messages get through from then on.

The challenge defeats spammers because virtually all junk mailers use bogus return addresses, and therefore can never correctly respond. But because the challenge demands a response, this type of tool can inconvenience the sender of the message. Afraid that a sales lead will e-mail someone else rather than go through the hassle of a response, businesspeople dread these kinds of tools even more than spam.

But for home users who bridle at the restrictions of a permission-based filter, and who dislike the hassle of training a rules-based filter, challenge/response filters can cut 100 percent of spam--and that's a diet that no one would mind sticking to.

Spam Tips: Let Your Internet Provider Do the Blocking

Don't want to pay for spam filtering software? You might want to consider using the mail scrubbing tools that your ISP provides instead. If you use a major national ISP, the company's filter can screen out as much junk as the average desktop filtering app--if you take the time to train it. Here's a roundup of tools and techniques that the national ISPs use to remove spam from their customers' in-boxes.

EarthLink's SpamBlocker lets the customer choose how aggressively the ISP should filter spam. The lowest level lets everything through; the middle level uses Brightmail. (In InfoWorld tests, Brightmail's spam-filtering accuracy averaged 96 percent, with a false-positive rate of less than 1 percent.) The highest level permits only messages from senders in the user's address book.

United Online (the parent company of NetZero, Juno, and BlueLight) uses content filters (tools that scan the subject and body of messages for keywords), whitelists, blacklists, and a spam reporting system.

MSN Premium and Plus and Hotmail use Brightmail as well as a Microsoft filtering tool that you train over time. MSN Premium and Plus offer five levels of filtering for junk e-mail; the highest level allows an MSN customer to get e-mail only from people listed in the user's MSN address book.

SBC Yahoo's Anti-Spam Resource Center provides a personalized filter that augments systemwide filtering. SBC uses a Real-time Blackhole List, detects and blocks spam servers not yet on the RBL, and blocks any computer on SBC's DSL network that sends mass amounts of e-mail.

At press time, DSL ISP Speakeasy said that it was switching to a tool based on the open-source SpamAssassin (which, in tests for InfoWorld, cleaned out spam e-mail with 93 percent accuracy).

Cox Communications, a cable ISP, uses filters (visit Cox.net for more information) based on Brightmail.

Cable ISP Adelphia Communications scans its network for customer PCs that are vulnerable to takeover by spammers. It also filters with Brightmail and uses RBLs to cut off spam-friendly networks.

Comcast Communications, another cable ISP, uses a combination of several commercial products that the company rep declined to name. In addition, the company probes its network for spam zombies--PCs that are being controlled by spammers without the owners' knowledge. If the probe discovers a zombie PC, Comcast notifies the customer first; then if the problem doesn't get fixed, it cuts off Internet service temporarily until the enslaved PC can be dezombified.