Featured Research

from universities, journals, and other organizations

Study details shortage of replication in education research

Date:
August 14, 2014
Source:
American Educational Research Association
Summary:
Although replicating important findings is essential for helping education research improve its usefulness to policymakers and practitioners, less than 1 percent of the articles published in the top education research journals are replication studies, according to new research.

Although replicating important findings is essential for helping education research improve its usefulness to policymakers and practitioners, less than one percent of the articles published in the top education research journals are replication studies, according to new research published today in Educational Researcher (ER), a peer-reviewed journal of the American Educational Research Association.

Related Articles


"Facts Are More Important Than Novelty: Replication in the Education Sciences," by Matthew C. Makel of Duke University and Jonathan A. Plucker of the University of Connecticut, analyzes the complete publication history of the current 100 education journals with the highest five-year "impact factor" (an indicator of how often a given journal's articles are cited in other scholarly work), finding that only 0.13 percent of published articles were replications.

Contrary to findings in medicine, but similar to psychology, the majority of education replications successfully replicated the original studies. Nearly 68 percent (67.4 percent) of the replications successfully replicated the findings of the original study, 19.5 percent had mixed results (supporting some, but not all, findings), and 13.1 percent failed to replicate any of the original findings.

However, replications were significantly less likely to be successful when there was no overlap in authorship between the original and replicating articles. Replication studies that were conducted by completely new research teams were found to be successful 54 percent of the time. When replications were conducted by the original authors in the same publication as the original findings, 88.7 percent were successful. When replications were in a new publication, but at least one author was on both the original and replicating studies, 70.6 percent of replications were successful.

Overall, about half (48.2%) of all published replications are done by the research team that completed the original study.

The shortage of replication studies receives attention across research fields. A 2005 review of highly cited medical publications found that only 44 percent of replications produced results similar to the original study. A 2012 study by the same research team of the publication history of the top 100 psychology journals found that only 1.07 percent of publications were replications.

Although the percentage of education replications is quite small, it is increasing. Currently, about one in 500 studies are replications, an increase from one in 2,000 in 1990.

"The desire to distinguish 'truth from nonsense' is a constant struggle within science, and the education sciences are no exception," said co-author Jonathan A. Plucker. "The need to increase replication is apparent and permeates all levels of education research, to provide both confidence in our collective work and disincentives for misconduct."

"If education research wants to be relied upon in the real world, conducting independent replication is essential," said co-author Matthew C. Makel. "Replications of important research findings will lead to stronger and more accurate policy and practice recommendations."

The authors identified eight reasons, or biases, that help explain the lack of replication in the education research field. These include submission bias, funding bias, editor/reviewer bias, journal publication policy bias, hiring bias, promotion bias, journals-analyzed bias, and "novelty equals creativity" bias. (See page 3 of the study for details about these factors.)

The authors recommend several steps for increasing the rate of replication in education research. These include revising journal editorial policies to explicitly encourage submission of replication studies, creating a new article type in journals for replications, adopting universal standards for conducting replications, and introducing replication training into graduate and undergraduate student projects.


Story Source:

The above story is based on materials provided by American Educational Research Association. Note: Materials may be edited for content and length.


Journal Reference:

  1. M. C. Makel, J. A. Plucker. Facts Are More Important Than Novelty: Replication in the Education Sciences. Educational Researcher, 2014; DOI: 10.3102/0013189X14545513

Cite This Page:

American Educational Research Association. "Study details shortage of replication in education research." ScienceDaily. ScienceDaily, 14 August 2014. <www.sciencedaily.com/releases/2014/08/140814124650.htm>.
American Educational Research Association. (2014, August 14). Study details shortage of replication in education research. ScienceDaily. Retrieved October 24, 2014 from www.sciencedaily.com/releases/2014/08/140814124650.htm
American Educational Research Association. "Study details shortage of replication in education research." ScienceDaily. www.sciencedaily.com/releases/2014/08/140814124650.htm (accessed October 24, 2014).

Share This



More Science & Society News

Friday, October 24, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Ebola Fears Keep Guinea Hospitals Empty

Ebola Fears Keep Guinea Hospitals Empty

AP (Oct. 23, 2014) Fears of Ebola are keeping doctors and patients alike away from hospitals in the West African nation of Guinea. (Oct. 23) Video provided by AP
Powered by NewsLook.com
Despite Rising Death Toll, Many Survive Ebola

Despite Rising Death Toll, Many Survive Ebola

AP (Oct. 23, 2014) The family of a Dallas nurse infected with Ebola in the US says doctors can no longer detect the virus in her. Despite the mounting death toll in West Africa, there are survivors there too. (Oct. 23) Video provided by AP
Powered by NewsLook.com
Academic Scandal Shocks UNC

Academic Scandal Shocks UNC

AP (Oct. 23, 2014) A scandal involving bogus classes and inflated grades at the University of North Carolina was bigger than previously reported, a new investigation found. (Oct. 23) Video provided by AP
Powered by NewsLook.com
Chameleon Camouflage to Give Tanks Cloaking Capabilities

Chameleon Camouflage to Give Tanks Cloaking Capabilities

Reuters - Innovations Video Online (Oct. 22, 2014) Inspired by the way a chameleon changes its colour to disguise itself; scientists in Poland want to replace traditional camouflage paint with thousands of electrochromic plates that will continuously change colour to blend with its surroundings. The first PL-01 concept tank prototype will be tested within a few years, with scientists predicting that a similar technology could even be woven into the fabric of a soldiers' clothing making them virtually invisible to the naked eye. Matthew Stock reports. Video provided by Reuters
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:

Strange & Offbeat Stories


Science & Society

Business & Industry

Education & Learning

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins