Analysis Of Beverage Studies Shows Pervasive Bias; Nutrition Studies' Conclusions Tied To Funding Source

David Ludwig, MD, PhD, the study's senior author and director of the Optimal Weight for Life (OWL) program at Children's Hospital Boston, believes that bias in nutrition studies may have far greater effects than bias in pharmaceutical studies. Not only do the findings of nutrition studies receive frequent media attention, but they influence governmental and professional dietary guidelines, the design of intervention programs, and FDA regulation of health claims on foods and beverages.

"We don't all take drugs, but we eat every day," Ludwig says. "If the science base is compromised by conflict of interest, that's a top-order threat to public health."

Because the researchers focused their analysis on soft drinks, juice and milk, they aren't sure whether their findings extend to nutrition studies as a whole. "We chose beverages because they represent an area of nutrition that's very controversial, that's relevant to children, and involves a part of the food industry that is highly profitable and where research findings could have direct financial implications," Ludwig says.

The researchers began by conducting a Medline search of all existing scientific literature about soft drinks, juice and milk published during a five-year period (1999-2003). They retrieved 538 articles, of which 206 were eligible for analysis. Eligible articles had to look at health outcomes or disease markers, had to involve humans or human tissue, had to be classifiable as an interventional or observational study or a scientific review, and had to explicitly state the beverages' effects on health measures. Of the 206 eligible articles, 111 declared financial sponsorship.

To guard against bias, the studies were analyzed independently. One investigator selected the articles for inclusion according to pre-established criteria. Another two investigators, who had no knowledge of the financial sponsors, and who were not told the article's author, title or journal of publication, classified the articles' conclusions as "favorable," "neutral" or "unfavorable." A fourth investigator, who had no knowledge of the conclusions, determined the funding source (22 percent were funded entirely by industry, 47 percent had no industry funding, and 32 percent had mixed funding) and classified articles as to whether a favorable finding would be beneficial, negative or neutral to its funder's financial interests.

When analyzed statistically, article conclusions were significantly related to funding source. Interventional studies with all-industry funding were much less likely to have unfavorable conclusions than those with no industry funding (0 vs. 37 percent). Among all types of studies, comparing all-industry versus no-industry funding, the odds ratio for having a favorable versus unfavorable conclusion was 4.37, increasing to 7.61 when beverage type, publication year and examination of authors' personal conflicts of interest were taken into account.

The researchers note that during their five-year study, the overall proportion of papers declaring their funding sources increased, as journals tightened their disclosure requirements. However, taking publication year into consideration didn't weaken the study's findings. "This isn't an effect that's disappearing," Ludwig says.

The bias may take several forms, Ludwig adds. Although false data could be present in some studies, more likely the biases reflect subtle manipulations, such as framing the questions in a way that make the results more favorable to a sponsor. In other cases, papers with unfavorable results may simply go unpublished.

The study's first author was Lenard Lesser, MD, a Children's research assistant now in the Tufts University Family Medicine Residency. Co-authors were Cara Ebbeling, PhD, of Children's Hospital Boston, Merill Goozner of the Center for Science in the Public Interest (Washington, DC), and David Wypij, PhD, a biostatistician at the Harvard School of Public Health.

The study was funded by the Charles H. Hood Foundation (Boston, Mass.) and the Department of Medicine at Children's Hospital Boston.