Jimmy Wales' Search Project Gets a Grub Stake
Posted by: Rob Hof on July 27, 2007
Wikipedia founder Jimmy Wales is moving another step closer to creating an open alternative to Google’s search engine. Announced last December, the Search Wikia project is committed to “fix Internet search by working to free the judgment of information from invisible rules inside an algorithmic black box.” That would be, um, Google. And other search engines, such as they are. The notion is to use human collaboration like that which built Wikipedia (a separate project, by the way) to come up with better search results.
This morning, Wikia, a company Wales also cofounded, acquired Grub. It’s a search engine technology company bought by Looksmart in 2003, that uses potentially millions of computers on the Net to crawl and the Web and analyze the results—like the SETI@Home project, which uses volunteers’ idle computer time to crunch data in the search for extra-terrestrial intelligence. According to the Wikipedia entry on Grub, some Web sites—including Wikipedia itself—had problems with the way it did its work. But Wales says Wikis is releasing Grub under an open-source license so anyone can work on it and improve it.
Basically, Grub adds to Wikia’s search project—at least in concept—the heavy computational lifting needed to crawl the Web. “This is a key piece of the puzzle,” Wales told me. He says Wikia will make available the results of the crawl, the index, as well. By year-end, he hopes to have a public search site ready, though he says it won’t be very good until volunteers keep hammering away to improve it. Chris Sherman at Searchengineland, who had trouble trying out the Grub software, hopes developers will start hammering soon.
As Wales admits, Wikia is “something of a political statement” that transparency works better than secrecy. He also thinks search has “hit something of a plateau” and needs a new approach. Not least, he hopes the search project, which will be ad-supported, eventually will become a big business if it works.
Will it? “I don’t know,” Wales concedes, though clearly he hopes it will. Whether or not it does, and we won’t know that for a good while, it’s encouraging that he and others are looking to push the boundaries of search. Good as Google and others can be, search is not yet a solved problem.