Bad science equals bad diets
Brian Wansink’s fall indicts an entire type of nutrition research.
Aspectacular case of sloppy science came crashing to a close last month. Cornell University’s Brian Wansink, a worldrenowned scientist who seized headlines with his research on American eating habits, had many of his papers retracted and resigned from his professorship. Wansink’s fall is not just the tale of a single scientist gone astray. It is, instead, an indictment of an entire type of nutrition science that has led to mistaken dietary advice dispensed to Americans for decades.
Wansink’s misdeeds played out in Cornell classrooms and nutrition journals and, as crucially, around American dinner tables. He had a central role in all our diets. From 2007 to 2010, he served as executive director of the U.S. government’s Dietary Guidelines, which set the standard for healthy eating for the nation. Under Wansink, the guidelines shifted to be ever-more based on the same type of easily manipulated, weak observational data he produced in his lab. A new iteration of the guidelines, under a different director, was issued in 2015, yet the reliance on weak data has remained the same.
Wansink’s research depended on observational studies, which can yield only associations, such as “coffee is associated with cancer.” These are at best suggestions of hypotheses, and they nearly always fall far short of demonstrating cause and effect. To show causation — that coffee causes cancer, for example — a real experiment, or clinical trial, is needed. For nutrition policy, however, we have rarely required that caliber of research.
Worse, the associational data in nutrition studies are particularly unreliable because the studies depend on self-reported answers on dietary questionnaires with such queries as: How many cups of pasta did you consume weekly for the last six months? Or, how much did you enjoy that last slice of pizza? Studies have long shown that people misrepresent what they eat — or they simply can’t remember.
John Ioannidis, a Stanford University professor and evidence-based medicine expert, recently wrote that, given all of the problems with this kind of nutrition research, “Reform has long been due.” The claims of this weak science, when tested properly by rigorous clinical trials, have been shown in two analyses to be correct only 0% to 20% of the time. This means that 80% to 100% of the time, they’re wrong.
These kinds of odds may be relatively harmless when it comes to studying our intake and enjoyment of pizza, as Wansink did. Yet for the Dietary Guidelines, much more is at stake.
The guidelines are surprisingly powerful: They drive choices for school lunches, feeding programs for the elderly, hospital food and military rations, as well as inf luencing the advice dispensed by doctors, nurses, dieticians and nutritionists. If the guidelines are off or downright wrong, the potential impact on our epidemics of obesity, diabetes and neurological diseases is devastating.
Critics have argued that the guidelines, launched in 1980, were based on shaky science before Wansink got involved, but his guidance arguably drove them off the rails. His 2010 guidelines turned the recommendations away from a focus on food groups and toward dietary patterns. This sounds reasonable enough, yet the fundamental flaw lies in the evidence base for the patterns.
I examined each study cited by the guidelines’ advisory committee to support the recommendations, and the results were astonishing. The government’s three recommended Food Patterns — “U.S.-Style,” “Mediterranean” and “Healthy Vegetarian”— were supported by fewer than a handful of truly rigorous trials. Even the government’s expert panel noted the deficiencies.
A single Mediterranean-diet trial, on only 180 people, was cited to demonstrate that any of these patterns could produce significant weight loss. The advisory committee deemed there was “limited” or “insufficient” evidence to show that the diets could combat diabetes. And the support for claims that the diets could prevent heart disease was ambiguous at best (blood pressure was the only risk factor that improved in quite a few studies, and in others, both “bad” and “good” cholesterol dropped, implying mixed outcomes overall).
The key, large clinical trial cited to claim cardiovascular benefits for the Mediterranean diet was recently retracted and reissued, raising serious doubts about its basic reliability. Meanwhile, the government’s review of the Healthy Vegetarian diet concluded that the evidence for any disease-fighting powers of this diet was “limited”— the lowest rank given for available data.
The lack of rigorous experimentation is matched by the data’s lack of transparency. In retracting Wansink’s papers, the editors of the Journal of the American Medical Assn. said their primary reason was that the data could not be confirmed because the original dietary surveys had been thrown out. Much the same problem dogs the country’s largest observational nutrition study, funded by the National Institutes of Health and run by Harvard’s T.H. Chan School of Public Health. Its findings are among the most highly cited in the Dietary Guidelines’ evidence base, yet Harvard does not publicly disclose its data, making those findings also impossible to confirm.
Defenders of the current guidelines’ argue that the urgency of our obesity and diabetes epidemics virtually demands that we march forward with recommendations based even on imperfect data. Yet mistakes of the past, such as the now-jettisoned caps on dietary cholesterol and total fat, show the real harm of rushing to create policy based on weak science. Most people still don’t even know about these reversals in our dietary guidelines — that the weight of evidence has now shifted to sugars and refined carbohydrates as the more likely dietary culprits. Un-learning diet rules once they have been learned proves to be extremely difficult.
Wansink’s downfall reveals the corrosive powers perpetually undermining good science in nutrition. Observational studies are supposed to be exploratory work, hardly ready for prime time. Yet the pressure to publish, the reality that media coverage drives grant making, and that these both drive the professional trajectory of researchers, push scientists like Wansink and others to oversell their results.
When imperfect data shape headlines, and the entire Dietary Guidelines, the losers are science itself and, quite clearly, the public health.
Nina Teicholz is a science journalist and executive director of the Nutrition Coalition, which is dedicated to evidence-based nutrition policy.