Not only have I spent a lot of time over the past six years with search engines, but I spent hours comparing Google specifically to GoTo.com, a search engine that lists results based on paid listings. If you search for "kitchen gadgets" and Chef's Resource is a GoTo advertiser, it'll come out at or near the top. (It is, and it did.) In fact, GoTo will list lots of advertisers, ranked by the amount they bid per user who clicks through to the advertiser's site.
Chef's Resource, for example, pays 35 cents a click. Yahoo, which pays 31 cents a click, is listed second. When there are no advertisers -- such as when you're searching for something sufficiently uncommercial that none will pay to be atop the listing -- GoTo uses a conventional search technology based on a proprietary algorithm.
By contrast, Google firsts bring up the site that, based on the number of links to it from other pages (and the number of links to those other pages from still more sites), has essentially been voted the Web's most popular page about kitchen gadgets. As it turns out, Google's top kitchen-gadget site is Kitchen-Classics.com, based in Phoenix. Google also sells to the highest bidder what it calls "sponsored links," but its screen is laid out so the advertisements are separated from the results generated by the search engine itself.
DIRECT CHALLENGE. The inspiration for the trial was classic: An editor at BusinessWeek received a study from New York-based researcher Jupiter that found people completed searches faster using pay-for-listing search engines like GoTo. This meant consumers got the most relevant listings right away using GoTo's approach, Jupiter argued. It was a direct challenge to Googlemania that demanded an investigation.
So we constructed a bakeoff between Google and GoTo to see what each was good for. The specific test was to run four searches on each and see what came up.
First, I wanted to find information about rebates offered in New York State on energy-efficient air conditioners. I also wanted to learn about the Environmental Protection Agency's standards on noise pollution -- there's a really loud fan in the courtyard of my building, and I'm itching to file a complaint. Third, I was curious to find out how the nominations for the Pulitzer Prize for poetry are conducted. Finally, I needed help mapping out some changes in my diet, since I'm doing a lot of running.
STICKY PROBLEM. In a short comparison between GoTo and Google, I failed to come up with the same results as Jupiter's survey. I found Google a little bit faster than GoTo. But I did relearn the importance of jumping around between search engines, because each has its strengths and weaknesses. My days of sticking it out with Google until I find the results I want are over.
Why? Because when you're looking up something online, time is not the only thing that counts. Part of the experience is the frustration of being subjected to exactly the wrong list of results. Another is the disturbing feeling that you're missing out on other good options.
In the end, neither search engine emerged as obviously superior. Both were good, quickly offering up clean lists of results. GoTo was superior on two of my four test searches. My takeaway: Don't be loyal to either one. If you don't find what you're looking for pretty quickly on Google or GoTo, you should jump ship and try the other rather than indefinitely fiddling with new search terms.
DOWN TO SPECIFICS. It took a total of 11.5 minutes to find the results I wanted on Google, while GoTo gave me what I needed in 13. Those results don't count the time I then spent looking through the suggested sites to see if they were helpful and had the information I wanted. I counted only how long it took the search services to dish up the results and for me to locate one that looked promising. I didn't accept results that gave me a listing for a broadbrush Web site, such as www.epa.org. I wanted links that would take me more directly to the specific information I needed.
GoTo was superior at delivering specific articles I wanted. One thing that struck me about Google was the dearth of plain old article listings. Google seems to default more to the main sections of sites, while GoTo dishes up listings that hit the mark more directly.
For instance, it showed me a link to a really interesting article at Men's Health magazine's site about what happens when you "hit the wall" of exhaustion late in a marathon and how to prepare for (and prevent) it. That was really helpful in planning a training diet.
DEBUNKING THEORIES. Another drawback I found with Google was that it seems to take search terms very literally. It took me six minutes and six variations on "EPA, sound, pollution, noise, and rules" before I finally found the Noise Pollution Clearinghouse, which provides links to regulatory information. GoTo seems to set up its listings more liberally. It took me two minutes and just one search on "EPA sound pollution" to find the exact same site.
However, Google paid off for the search about air-conditioner rebates. That's probably because an entire site, Getenergysmart.org, is devoted to the rebate program, and lots of other sites refer their visitors to it. It took just a minute and a half using Google to find that site, while GoTo took three minutes.
The results contradicted my belief that Google was miles better than any other search service, and it debunked Jupiter's study saying paid search services like GoTo return more relevant results. It also shot down one of our own hypotheses, which held that GoTo would be at a huge disadvantage in searches on noncommercial issues, for which no advertisers would bid for places near the top of its listings. Neither capitalism nor technology can overcome a simple fact: There's lots of information to sift through online. Green covers the Internet and e-commerce for BusinessWeek from New York