I often think spam is a great example of everything that can go wrong with technology in general and egovernment in particular: the Internet and its many facilities - MUDs, IRC, Usenet, email, search engines, IM, blogs, and wikis were all designed with the same anarchic optimism in mind. (Anarchy in the old sense, in which people eschew laws, rules, and governments and choose to have faith that others will be good actors, hence optimistic.) And along come spoilers who turn the system to their own advantage and are not interested in community standards.

This is my writeup for the Reg of last Friday’s antispam conference at MIT. This was a small gathering - maybe 30 or 40 people - but it included representatives of several household name ISPs as well as researchers who had traveled from as far as Italy and Croatia to present papers. Someone (who prefaced every single thing he said with “You can’t quote me”) commented that he thinks this type of small, brainy gathering is far more likely to come up with solutions than the better-known industry forums that are much bigger. In general, he says, those fora come up with just more of the same.

Among things that didn’t get into the Reg writeup:
- a paper (Angela Blanco from the University of Salamanca) carefully demonstrating that multiple classifiers really are better than one (which sounds obvious but still needs to be proved)
- a paper (Amanda Watlington) on the difference between search engine marketing and search engine spam
- a presentation (Six Apart) on splogs and blog spam generally
- a proposal for reputation technologies (this guy didn’t get very far because early on he dissed Spamhaus, which most people agree is hugely helpful, and which many major ISPs rely on; his company wants to sell reputation management as a service)
- a paper (Alberto Trevino, Brigham Young University) on header relay detection (this paper was roundly criticized by Eric Allman, creator of Sendmail, on the grounds that it violates RFC2821 and has other significant problems such as relying on people to read bounce messages, which themselves are usually spam, so people have stopped reading them)
- a paper proposing a way to modify SPF so that legitimate remailing (eg, Blackberry) would not be blocked; the presenter believes that this sort of problem is blocking wider adoption of SPF
- Bitdefender on multiple filters

The papers should all be up soon (if they’re not already) here. (Good practice: they went around collecting the presentations from the speakers onto a USB key before letting them leave.)

Some things about this:
1) Spam is a hugely intractable problem
2) There is no single solution (because there is no one type of spam and no one motive or modus operandi of the spammers)
3) Government action in the form of laws and regulation is not much help (although the existence of laws may help prosecutors once a spammer is actually caught - yet that does not result in any reduction of the problem).
4) Every system that people use to communicate with each other is vulnerable.
5) Services involving public collaboration must be designed with the understanding that they will be vulnerable to spam once they reach a certain prominence or size. (Law blogs last year were estimating the readership at which comment spam became an issue at around 10,000; I have a blog that’s hardly used yet gets comment and trackback spam - because my site’s page rank is 6, which is fairly high for an individual Web site). Things like Google’s Adsense and Amazon associates have created economic motives for search engine spam splogs, yet no one would have expected them to contribute to the spam problem. In the case of wikis, you’d think spam would be a problem but the ability to roll back changes tends to obviate a lot of it; abuse in the form of content spoilage seems to be the more acute problem with bigger wikis, to which the only answer is human moderators.

wg