When Yahoo! Inc. (YHOO) embarked on a mammoth redesign of its front page last year, almost every department at the company suddenly had an opinion. Naturally, conflicts arose. "Sales would want the ad front and center, and the Yahoo Mail person would say: That could potentially cannibalize my business,'" recalls Tapan Bhat, Yahoo's vice-president for front doors, who oversaw the project.
To avoid design by committee, Yahoo deferred almost every decision to an impartial judge: data generated by users' clicks. "We have this culture of data," Bhat explains. "It is the biggest enforcer of honesty." If sales wanted an ad smack dab in the front page's prime real estate, the company would whip up a page to those specifications, serve it to actual users, and record their clicks. If traffic increased, great. If not, it was back to the drawing board. The refreshed home page went live in September. Now, with a multimillion-dollar ad campaign to promote the redesign under way, the company is keen to see whether it has truly created a page based on what users like rather than what Yahoo wants.
Getting the home page right is crucial for a portal like Yahoo, especially since the company's advertising revenue has been slowing. Yahoo announced on Sept. 20 that third-quarter gross profit would probably be in the bottom half of its forecast of $925 million to $1 billion, representing year-over-year growth of 14% to 20%. That's slower than the last two quarters by one-third to one-half.
"The Yahoo home page is a very important driver of revenue," notes James Friedland, an analyst at New York investment bank Cowen & Co. (COWN) He estimates that 40% to 45% of Yahoo's $4.7 billion in annual sales comes from display ads such as banners, which appear on pages that users often reach through the front page. Says Bhat: "Any link on the front page has huge ripple effects all throughout Yahoo, and it affects our revenue."
Yahoo made a commitment to harnessing its trove of user clicks in 2004 when it acquired DMX Group, a data mining consultancy founded by former Microsoft Corp. (MSFT) researchers. Now called Strategic Data Solutions (SDS), the department has a daunting task: combing through the 10 terabytes of data that Yahoo users generate daily by clicking links (the equivalent of all the text in the Library of Congress), plucking out the relevant bits, compressing it, and storing it. So far, Yahoo has enough user data to fill more than 1,000 Libraries of Congress.
Of course, all that information would be useless without a way to make sense of them. Before Yahoo bought DMX Group, a simple test of how users interact with a page required help from technologists and a month of preparation. Now nearly all employees have access to easy-to-use software tools that can run tests over a few hours or days. Along with providing the tools, the SDS department has worked to spread the gospel of data. "We say: Use data to make decisions. Don't make decisions based on a fad or what your competitors are doing,'" says Bassel Ojjeh, vice-president for SDS.
BRITNEY VS. BRITTANY
To help drive that message home, the SDS team this spring invited Stanford University business professor Robert I. Sutton to speak at Yahoo. Sutton and his Stanford colleague Jeffrey Pfeffer are co-authors of Hard Facts, Dangerous Half-Truths, and Total Nonsense: Profiting from Evidence-Based Management. The book owes a debt to evidence-based medicine, an approach in which doctors use recent clinical research to inform decisions about treatment. Applied to business, the approach requires managers to base decisions on facts rather than opinions. "It's this attitude that instead of sitting around having debates about what will work and what won't, let's run some experiments and learn," Pfeffer says.
What Yahoo learned often belied initial impressions. Throughout the redesign, the company used a blend of focus groups, one-on-one interviews, test pages, and data mining. "What people say they want isn't always what they actually click on," Bhat says. In focus groups, users consistently said they wanted serious world news. "I don't want Britney Spears anywhere on my page," Bhat recalls one user saying. "What if my boss came by and saw?" But when Bhat's team studied users' clicks, world news got little attention, while Britney Spears stories ranked among the most heavily trafficked.
The mixed messages led to important insights. In the end, Yahoo kept world news prominent on the front page because users feel secure knowing that it's easily accessible, even if they don't often click it. Conspicuous placement also went to entertainment, which draws heavy traffic from people seeking a diversion at work. By contrast, seemingly work-related content such as finance gets ample use in the evening when people pay bills and manage personal portfolios.
Another gem unearthed through data mining: Small changes can make a big difference. The redesign team was excited about a new feature called Personal Assistant, which lets users hover their pointers over icons to see preview boxes of content such as e-mail. "We knew this was going to be the wow' element of the page," Bhat says. But the data showed that users were less than wowed. Turns out the preview boxes opened too quickly, an unusual peeve in this caffeinated, wired world. So the team began fiddling with the speed at which the preview boxes appeared and introduced a slight delay. Bingo.
Although Yahoo's front-page redesign is finished, the testing is not. "There's always some test running," Bhat says. "It's part of our DNA." Now if only Yahoo could collect as much data about its advertisers' spending habits.
By Justin Hibbard