The study was published by the respected Journal of the American Medical Assn. (JAMA). But before you rush out for fast food, ask a few simple questions. What did the study actually examine? Do its data support such a sweeping conclusion? We often overreact to studies that are more limited and uncertain than we realize. One startling fact: Fully one-third of all dramatic new findings simply don't hold up in later studies. Remember hormone replacement therapy? Once presumed to cut heart disease risk, it was later found to be dangerous -- and now doctors are questioning those supposed risks.
Among other things, the new diet report shows how research dollars can be won for questionable studies. In the early 1990s, when the study was proposed, prominent researchers such as Harvard's Dr. Walter Willett argued that it wasn't worth doing. Reviewers at the National Institutes of Health turned it down for funding. But Dr. Bernadine Healy, appointed as NIH director in 1991, lobbied relentlessly for support for the suite of studies that make up the Women's Health Initiative. "It was an end-run through Congress that got it approved," Willett recalls.
Based on its design, the trial had little chance of making major advances in the science of nutrition. It set out to compare women eating normal diets with women asked to eat less fat and more fruits and vegetables. But the two groups' diets really didn't differ all that much. Those with "healthier" diets cut calories from fat by only 8.2% compared with the normal group and ate only 1.1 additional servings of fruits and vegetables a day. That's too small a difference to expect health benefits.
What's more, scientists now know that the type of fat we eat is generally more important than the amount. Monounsaturated fats, such as those in olive oil, or omega-3 fats found in fish are healthier than saturated fat, as in butter or beef. Worst of all are so-called trans fats, which are created by adding hydrogen to natural fat. Trans fats are widely used in everything from margarine to cookies, and have been linked with increased risks of heart disease. By focusing just on total fat intake, the study missed a chance to tease out the effects of these different types of fat. "It's really important not to go away with the message that diets don't matter," says Willett.
There are larger lessons as well about how science works. The detailed methods of a study are crucial, and their limitations are often glossed over. When Dr. Steven Woloshin of Dartmouth Medical School looked at press releases trumpeting various scientific articles, "we found that 77% failed to mention the study's limitations," he says. And up to half of all articles exaggerated the actual benefit or risk being studied.
One common distortion is to present the effects of a treatment or intervention in relative terms, such as saying that a particular diet or drug cut the death rate in half. That sounds huge, but it might mean that there was only one death per million people instead of two, which is a reduction in actual risk that's too small to matter. All this helps explain why science so often seems to be flip-flopping. And there's no mystery about why results are often exaggerated: Researchers, medical centers, and scientific journals are all competing for funds and publicity, just like everyone else.
The truth is, science is often less definitive than people think. Even when a study does find a clear benefit or danger, the results may not be applicable to the real world, where people have more complicated health issues and medicine isn't so black and white. There's a long list of treatments and medical interventions, from bypass operations to back surgery, where the benefits may not be as great as once thought. What does this mean for diet? "We don't want to be morbidly obese or too skinny," says Dr. Nortin Hadler, professor of medicine at the University of North Carolina at Chapel Hill, "and it's pretty compelling that between those two extremes, we can't measure many differences." So try to stay fit, keep your weight down, aim for a relatively healthy diet, and don't worry too much about the latest headlines in the medical journals. By John Carey