Featured Research

from universities, journals, and other organizations

Bias pervades the scientific reporting of animal studies, research suggests

Date:
July 16, 2013
Source:
Public Library of Science
Summary:
A new study suggests that the scientific literature could be compromised by substantial bias in the reporting of animal studies, and may be giving a misleading picture of the chances that potential treatments could work in humans.

A new study published in the open access journal PLOS Biology suggests that the scientific literature could be compromised by substantial bias in the reporting of animal studies, and may be giving a misleading picture of the chances that potential treatments could work in humans.

Testing a new therapeutic intervention (such as a drug or surgical procedure) on human subjects is expensive, risky and ethically complex, so the vast majority are first tested on animals. Unfortunately, cost and ethical issues constrain the size of animal studies, giving them limited statistical power, and as a result the scientific literature contains many studies that are either uncertain in their outcomes or contradictory. A way around this limitation has been to conduct a "meta-analysis": scientists collect data from a large number of published studies on the same intervention, combine them using sophisticated statistical methods, and then end up with a much more solid basis on which to decide whether to proceed with human clinical trials.

In the new study, Konstantinos Tsilidis, John Ioannidis and colleagues at Stanford University examined 160 previously published meta-analyses of animal studies looking at potential treatments for a range of serious human neurological disorders (multiple sclerosis, stroke, Parkinson's disease, Alzheimer's disease and spinal cord injury). These meta-analyses covered 1000 original published animal studies comparing more than 4000 sets of animals. The authors' "meta-analysis of meta-analyses" used the most precise study in each meta-analysis as an estimate of the true effect size of a particular treatment. It then asked whether the expected number of studies had statistically significant conclusions. Alarmingly, the authors found that more than twice as many studies as expected appeared to reach statistical significance.

The authors suggest that rather than reflecting wilful fraud on the part of the scientists who conduct the original studies, this "excess significance bias" comes from two main sources. One is that scientists conducting an animal study tend to choose the method of data analysis that appears to give them the "better" result. The second arises because scientists usually want to publish in higher profile journals; such journals tend to strongly prefer studies with positive, rather than negative, results. Many studies with negative results are not even submitted for publication or, if submitted, either cannot get published or are published belatedly in low-visibility journals, reducing their chances of inclusion in a meta-analysis.

It is likely that the types of bias reported in the new PLOS Biology paper have been responsible for the inappropriate promotion of treatments from animal studies into human clinical trials. It also seems unlikely that this phenomenon is confined to studies of neurological disorders; rather this is probably a general feature of the reporting of animal studies.

The authors suggest several remedies for the bias that they have observed. First, animal studies should adhere to strict guidelines (such as the ARRIVE guidelines) for study design and analysis. Second, animal studies (like human clinical trials) should be pre-registered so that publication of the outcome, however negative, is ensured. Third, availability of methodological details and raw data would make it easier for other scientists to verify published studies.


Story Source:

The above story is based on materials provided by Public Library of Science. Note: Materials may be edited for content and length.


Journal Reference:

  1. Konstantinos K. Tsilidis, Orestis A. Panagiotou, Emily S. Sena, Eleni Aretouli, Evangelos Evangelou, David W. Howells, Rustam Al-Shahi Salman, Malcolm R. Macleod, John P. A. Ioannidis. Evaluation of Excess Significance Bias in Animal Studies of Neurological Diseases. PLoS Biology, 2013; 11 (7): e1001609 DOI: 10.1371/journal.pbio.1001609

Cite This Page:

Public Library of Science. "Bias pervades the scientific reporting of animal studies, research suggests." ScienceDaily. ScienceDaily, 16 July 2013. <www.sciencedaily.com/releases/2013/07/130716194952.htm>.
Public Library of Science. (2013, July 16). Bias pervades the scientific reporting of animal studies, research suggests. ScienceDaily. Retrieved April 23, 2014 from www.sciencedaily.com/releases/2013/07/130716194952.htm
Public Library of Science. "Bias pervades the scientific reporting of animal studies, research suggests." ScienceDaily. www.sciencedaily.com/releases/2013/07/130716194952.htm (accessed April 23, 2014).

Share This



More Science & Society News

Wednesday, April 23, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

UN Joint Mission Starts Removing Landmines in Cyprus

UN Joint Mission Starts Removing Landmines in Cyprus

AFP (Apr. 23, 2014) The UN mission in Cyprus (UNFICYP) led a mine clearance demonstration on Wednesday in the UN-controlled buffer zone where demining operations are being conducted near the Cypriot village of Mammari. Duration: 01:00 Video provided by AFP
Powered by NewsLook.com
Air Force: $4.2B Saved from Grounding A-10s

Air Force: $4.2B Saved from Grounding A-10s

AP (Apr. 23, 2014) Speaking about the future of the United States Air Force, Chief of Staff Gen. Mark Welsh says the choice to divest the A-10 fleet was logical and least impactful. (April 23) Video provided by AP
Powered by NewsLook.com
Nuclear-Level Asteroids Might Be More Common Than We Realize

Nuclear-Level Asteroids Might Be More Common Than We Realize

Newsy (Apr. 23, 2014) The B612 Foundation says asteroids strike Earth much more often than previously thought, and are hoping to build an early warning system. Video provided by Newsy
Powered by NewsLook.com
High Court to Hear Dispute of TV Over Internet

High Court to Hear Dispute of TV Over Internet

AP (Apr. 22, 2014) The future of Aereo, an online service that provides over-the-air TV channels, hinges on a battle with broadcasters that goes before the U.S. Supreme Court on Tuesday. (April 22) Video provided by AP
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:
from the past week

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins