Can a search engine be open-sourced?
Salon has up the story that Jimmy Wales, founder of Wikipedia, is running a project to create an open source search engine. The important point about it, it seems to me, is not whether it can chellange Google or any of the other entrenched search services, but whether it can *survive* - because if it can it might provide an important service in testing the honesty of the main search engines. As things are, as Wales says in that article, access to information stored online is gatekept by secret algorithms. Of course, smart marketing people spend a lot of time figuring out how to work the system to get their sites well up the page rankings. But as a check on these things I think the important thing is know what you *can’t* find.
I see the Salon comments revolve around whether a publicly known algorithm could protect the search engine from being gamed ruthlessly by spammers. Two points: 1) Google *is* gamed by those goddamned shopping aggregators that serve no useful purpose in life - maybe if the algorithms were known more people would be able to come up with ways to prevent that (or to filter them out as a third-party service); 2) does the crypto analogy apply?
wg

Leave a Reply