Science News
from research organizations

Poor transparency and reporting jeopardize the reproducibility of science

Date:
January 4, 2016
Source:
PLOS
Summary:
Reported research across the biomedical sciences rarely provides full protocol, data, and necessary level of transparency to verify or replicate the study, according to two articles publishing in PLOS Biology as part of a new Meta-Research Section, on Jan. 4, 2016. The authors argue that the information publicly available on reported research is in dire need of improvement.
Share:
FULL STORY

Reported research across the biomedical sciences rarely provides full protocol, data, and necessary level of transparency to verify or replicate the study, according to two articles publishing in PLOS Biology as part of a new Meta-Research Section, on January 4th, 2016. The authors argue that the information publicly available on reported research is in dire need of improvement.

Authors of one study, Shareen Iqbal from Emory University, John Ioannidis from the Meta-Research Innovation Center at Stanford (METRICS) and colleagues, analyzed a corpus of papers published between 2000 and 2014 to determine the extent researchers report key information necessary for properly evaluating and replicating published research, including availability of protocols, data, and the frequency of published novel or replication studies. The authors were surprised by the results: out of 441 articles drawn from across the biomedical literature, only one paper provided a full protocol and no paper made all the data available. The majority of studies didn't state funding or conflicts of interest and replication studies were very rare.

"We hope our survey will further sensitize scientists, funders, journals and other science-related stakeholders about the need to improve these indicators," the authors stated.

A related study, led by Ulrich Dirnagl and team at Charité Universitätsmedizin in Berlin, Germany, examined hundreds of published stroke and cancer research experiments and found that the vast majority don't contain sufficient information about how many animals were used. What's more, in many papers animals "vanished" over the course of the study. Using a computer model, the team simulated the effects of such animal loss on the validity of the experiments. They found that the more animals lost or removed, the shakier or more biased the experimental conclusions.

"The study began with an attempt to look at the robustness of findings in a handful of preclinical papers" explains first author Constance Holman, "but the sheer number of missing animals stopped us in our tracks." In human medicine, publishing a clinical trial without information about the number of patients, or how many dropped out or died over the course of a study would be unthinkable. But nobody had looked carefully at whether animal numbers are properly reported in basic research.

Billions of dollars are wasted every year on research that cannot be reproduced. The findings of these two studies join a long list of concerns about bias and reporting in basic research. However, they also establish ways in which research can become more transparent and potentially more reproducible.


Story Source:

Materials provided by PLOS. Note: Content may be edited for style and length.


Journal References:

  1. Stavroula Kousta, Christine Ferguson, Emma Ganley. Meta-Research: Broadening the Scope of PLOS Biology. PLOS Biology, 2016; 14 (1): e1002334 DOI: 10.1371/journal.pbio.1002334
  2. Shareen A. Iqbal, Joshua D. Wallach, Muin J. Khoury, Sheri D. Schully, John P. A. Ioannidis. Reproducible Research Practices and Transparency across the Biomedical Literature. PLOS Biology, 2016; 14 (1): e1002333 DOI: 10.1371/journal.pbio.1002333
  3. Constance Holman, Sophie K. Piper, Ulrike Grittner, Andreas Antonios Diamantaras, Jonathan Kimmelman, Bob Siegerink, Ulrich Dirnagl. Where Have All the Rodents Gone? The Effects of Attrition in Experimental Research on Cancer and Stroke. PLOS Biology, 2016; 14 (1): e1002331 DOI: 10.1371/journal.pbio.1002331

Cite This Page:

PLOS. "Poor transparency and reporting jeopardize the reproducibility of science." ScienceDaily. ScienceDaily, 4 January 2016. <www.sciencedaily.com/releases/2016/01/160104163155.htm>.
PLOS. (2016, January 4). Poor transparency and reporting jeopardize the reproducibility of science. ScienceDaily. Retrieved May 23, 2017 from www.sciencedaily.com/releases/2016/01/160104163155.htm
PLOS. "Poor transparency and reporting jeopardize the reproducibility of science." ScienceDaily. www.sciencedaily.com/releases/2016/01/160104163155.htm (accessed May 23, 2017).

MORE COVERAGE

RELATED STORIES