Jimmy Wales' Search Project Gets a Grub Stake

Posted by: Rob Hof on July 27, 2007

Wikipedia founder Jimmy Wales is moving another step closer to creating an open alternative to Google’s search engine. Announced last December, the Search Wikia project is committed to “fix Internet search by working to free the judgment of information from invisible rules inside an algorithmic black box.” That would be, um, Google. And other search engines, such as they are. The notion is to use human collaboration like that which built Wikipedia (a separate project, by the way) to come up with better search results.


This morning, Wikia, a company Wales also cofounded, acquired Grub. It’s a search engine technology company bought by Looksmart in 2003, that uses potentially millions of computers on the Net to crawl and the Web and analyze the results—like the SETI@Home project, which uses volunteers’ idle computer time to crunch data in the search for extra-terrestrial intelligence. According to the Wikipedia entry on Grub, some Web sites—including Wikipedia itself—had problems with the way it did its work. But Wales says Wikis is releasing Grub under an open-source license so anyone can work on it and improve it.

Basically, Grub adds to Wikia’s search project—at least in concept—the heavy computational lifting needed to crawl the Web. “This is a key piece of the puzzle,” Wales told me. He says Wikia will make available the results of the crawl, the index, as well. By year-end, he hopes to have a public search site ready, though he says it won’t be very good until volunteers keep hammering away to improve it. Chris Sherman at Searchengineland, who had trouble trying out the Grub software, hopes developers will start hammering soon.

As Wales admits, Wikia is “something of a political statement” that transparency works better than secrecy. He also thinks search has “hit something of a plateau” and needs a new approach. Not least, he hopes the search project, which will be ad-supported, eventually will become a big business if it works.

Will it? “I don’t know,” Wales concedes, though clearly he hopes it will. Whether or not it does, and we won’t know that for a good while, it’s encouraging that he and others are looking to push the boundaries of search. Good as Google and others can be, search is not yet a solved problem.

TrackBack URL for this entry: http://blogs.businessweek.com/mt/mt-tb.cgi/

Reader Comments

Mark

July 28, 2007 09:52 AM

That's a great project. Unfortunately, there is no search engine that allows you to search for video programs. If you search on Google you always get YouTube or Google Videos.

Sites like Xeoss and Joost are nice but there is
not much content.

wolf

July 30, 2007 03:30 AM

www.faroo.com goes even a step further: distributed crawling *and* distributed search.

Sylvio Drouin

August 9, 2007 12:20 PM

try blipcast for facebook....standalone site coming soon...we search dailymotion, youtube, metacafe and break

Post a comment

 

About

BusinessWeek writers Peter Burrows, Cliff Edwards, Olga Kharif, Aaron Ricadela, Douglas MacMillan, and Spencer Ante dig behind the headlines to analyze what’s really happening throughout the world of technology. One of the first mainstream media tech blogs, Tech Beat covers everything from tech bellwethers like Apple, Google, and Intel and emerging new leaders such as Facebook to new technologies, trends, and controversies.

Categories

 

BW Mall - Sponsored Links

Buy a link now!