The Perils of Market Research

It can be a powerful weapon in any company's strategic planning arsenal. But it can also backfire. Steve McKee offers five essentials to consider

By Steve McKee

My colleagues and I conduct quite a bit of market research, both for our own company and on behalf of our clients. We've seen it used both for good and for ill. Here are a handful of thoughts that can keep your research from causing you pain.

1. There are things you can measure and things you can't. Don't mix them up.

How much do you love your wife? What's the value of poetry? What is a life worth? Ask most people these questions, and you'll get a funny look—or get into a metaphysical discussion. Some things just can't be quantified. Yet in business, we often act as if everything can.

In a 2008 Wall Street Journal story, Adrian Van Hooydonk, director of design at BMW (BMW:GR), explained how the carmaker evaluates vehicle prototypes: "We don't use customer clinics. They will be judging it based on the world today. Design needs to look good in eight years' time. You can't ask a customer whether he will like the design of the car in 2018." Van Hooydonk and his team must be onto something, because BMW has arguably been the most stylish and best-performing car company of the past two decades.

Research can't predict the kind of cars we will be interested in five years from now, how an ad concept will be received three months from now, or what the next hit movie or popular fashion trend will be. Yet we still hold onto hope that somehow statistics and spreadsheets will enable us to foresee the future. They won't. They can't.

Of course, it is possible to track events that have already happened, and that can provide valuable information. Such things as purchase patterns and visit frequency are historical, concrete events subject only to the laws of forgetting (I may not remember how I heard of your product) and deceit (I may not want you to know that I saw your ad in my wife's Glamour magazine). By and large they can be reliably tracked. But when we try to quantify attitudinal attributes—or assume that the past will accurately predict the future—we can get into trouble.

2. Just because you can't measure it doesn't mean it's not real.

When was the last time you responded to a TV ad—or any form of advertising, for that matter? That, of course, depends on the meaning of the word "responded." It's easy to think of "response" strictly as a measurable action, such as an inquiry or purchase. But there are many different ways consumers respond to ads, from the concrete (Web visits, phone inquiries, transactions) to the more abstract (preference, likability, identification). And the higher-involvement the purchase decision, the more likely a brand will need to generate a number of abstract responses before it sees anything concrete.

In 2003, Advertising Age ran a provocative story about advertising ROI—or the lack of it. The publication said that according to research, "Media advertising does the worst job of any marketing discipline in proving return on investment, and network TV is the worst of those media…."

The study was fielded among leading national advertisers—people who ought to know. But what, exactly, did it demonstrate? Not that network TV didn't generate ROI, but that it was the hardest medium with which to prove ROI. That's a significant distinction.

If network TV didn't work for advertisers, there would be no network TV. As business people, we know that TV works because it works on us as consumers. We may not be able to measure it with precision, but anyone who recognizes the Mac vs. PC guys, the Aflac Duck (AFL), the E-Trade Baby (ETFC), or the Budweiser Clydesdales (BUD) can't deny it. Passing judgment based only on what's easily measurable is myopic.

Business Exchange

Track and share business topics across the Web.