The search giant is unfazed by competition from "natural language" upstarts, but users—and advertisers—may benefit once they get the hang of the new tool
How do you beat Google (GOOG) at its own game? Search engine upstart Powerset is betting on the wisdom of the crowds. As the latest step in one of the longer buildups to a product launch in Internet history, Powerset on Sept. 17 is taking the wraps off a new online community site called Powerset Labs. The company hopes the site will marshal thousands of people to help build and improve its search engine before it goes public next year.
Powerset is one of several dozen startups presenting themselves at TechCrunch40, a showcase being held in San Francisco on Sept. 17 and 18 by bloggers Michael Arrington, editor of TechCrunch, and Jason Calacanis, founder of the human-powered search startup Mahalo.com. But given the mounting anticipation since its plans became public last year, Powerset is likely to be one of the most closely watched.
Aiming for Advertisers
Ultimately, the 70-person San Francisco company aims to one-up Google by producing search results that are even more on the mark—and potentially, more valuable to advertisers. Google's search engine matches words within search queries with text in Web pages. It also tallies links between Web pages to highlight the ones that get linked to the most from other Web pages.
In contrast, Powerset's system will analyze the actual meaning of words and phrases that it indexes on the Web. It then will analyze the linguistic meaning of the query and find the best matches between the two—theoretically, at least, producing more meaningful results. "Our system reads every single sentence in every single document and extracts meaning from them," says Powerset Chief Executive Barney Pell. The method is based on what's known as natural-language processing, which Powerset developed partly in-house and partly through a licensing agreement with PARC, Xerox's (XRX) famous Palo Alto Research Center.
Using Powerset's method, a searcher could ask, "What companies did PeopleSoft acquire in 2002?" and potentially get more on-point search results than from typing in mere keywords such as "PeopleSoft acquisitions 2002." Using Google, that query also brings up mentions of Oracle's 2004 acquisition of PeopleSoft. Another advantage of the Powerset approach is it gets to the meaning of words, so queries for one word will fetch results that contain words which are synonyms. Google's keyword searching generally won't do that.
Skeptics and Followers
Powerset, funded by Peter Thiel's Founders Fund and Foundation Capital, has a lot of skeptics. In a screed last October, search expert Danny Sullivan, editor-in-chief of the online news site Search Engine Land, noted that no claims of the superiority of natural-language search have ever held up. And he disputed the idea that most people would rather ask questions than simply type in a few words, noting Google didn't train people to query that way but simply responded to the way users were already conducting searches. "Linguistics will not solve most search problems," adds Apostolos Gerasoulis, executive vice-president of search technology at Ask.com, the search engine unit of IAC/InterActiveCorp (IACI).
Powerset's not alone. New York-based startup Hakia also plans to field a search engine employing similar techniques. Still others are taking a different approach to tapping people's brains to improve search. Through his for-profit company Wikia, Wikipedia founder Jimmy Wales has kicked off Search Wikia, a project that aims to apply the same human collaboration to search that built Wikipedia into a powerful online encyclopedia (BusinessWeek.com, 7/27/07).
Powerset will harness people power to improve the engine itself. Enter Powerset Labs, or Powerlabs for short, a site where a community of search engine enthusiasts can try out the engine, view video demos, and contribute ideas to improve it. A video demonstrates how the new method works.
Comparable to Digg.com, where readers can vote stories up or down a page, Powerlabs features a "new idea" button at the bottom of each page, so people can write in ideas for better search methods or features. They also can earn "karma" points to get access to new demos and gain community stature. "The wisdom of the crowds can raise the best ideas to the top," says Scott Prevost, Powerset's director of product.
Crafting Natural-Language Queries
Still, Powerset faces a key challenge even beyond the considerable difficulty of indexing the entire Web and returning results at lightning-fast speed. Its search engine will require users to change how they search away from typing in keywords—which Powerset's Pell dismisses as "grunting pidgin"—and toward full phrases, sentences, and questions. The more people type, the more of an edge Powerset may have in divining meaning from queries and producing more relevant results than just from keywords. "Absolutely this requires a change in user behavior," says Pell. "People have to go and learn that."
Indeed, that's the other key reason for launching Powerlabs. For instance, one demo lets people plug words into boxes separated by subject, verb, and object, the key components of a sentence. The idea is to show people how the Powerset search engine finds results differently once people learn to craft more complete queries. Another demo has sample questions into which names or other words can be typed and compares the results to those of "the other guys," namely Google.
For now, the demos are based only on Wikipedia. Pell says that's to test the system before going to the enormous expense of indexing the entire Web. And for now, the general public can't take part. Powerset will register about 500 people a week drawing from those who have signed up at its Web site.
Google executives have said that natural-language search could be years away from practical use and that linguistic analysis hasn't produced notably better results so far, which Powerset disputes. At the same time, there's little doubt Google's search wizards are examining the possibilities and are positioned to take swift advantage if the technology pans out. But even if Google isn't threatened by the competition anytime soon, it's clear the search game is far from over.