Featured Research

from universities, journals, and other organizations

To teach scientific reproducibility, start young

Date:
February 28, 2014
Source:
Duke University
Summary:
In the wake of retraction scandals and studies showing reproducibility rates as low as 10 percent for peer-reviewed articles, the scientific community has focused attention on ways to improve transparency and duplication. A team of math and statistics professors has proposed a way to address one root of that problem: teach and emphasize reproducibility to aspiring scientists, using software that makes the concept feel logical rather than cumbersome.

The ability to duplicate an experiment and its results is a central tenet of the scientific method, but recent research has shown an alarming number of peer-reviewed papers are irreproducible.

A team of math and statistics professors has proposed a way to address one root of that problem by teaching reproducibility to aspiring scientists, using software that makes the concept feel logical rather than cumbersome.

Researchers from Smith College, Duke University and Amherst College looked at how introductory statistics students responded to a curriculum modified to stress reproducibility. Their work is detailed in a paper published Feb. 25 in the journal Technological Innovations in Statistics Education.

In 2013, on the heels of several retraction scandals and studies showing reproducibility rates as low as 10 percent for peer-reviewed articles, the prominent scientific journal Nature dedicated a special issue to the concerns over irreproducibility.

Nature's editors announced measures to address the problem in its own pages, and encouraged the science community and funders to direct their attention to better training of young scientists.

"Too few biologists receive adequate training in statistics and other quantitative aspects of their subject," the editors wrote. "Mentoring of young scientists on matters of rigour and transparency is inconsistent at best."

The authors of the present study thus looked to their own classrooms for ways to incorporate the idea of reproducibility.

"Reproducing a scientific study usually has two components: reproducing the experiment, and reproducing the analysis," said Ben Baumer, visiting assistant professor of math and statistics at Smith College. "As statistics instructors, we wanted to emphasize the latter to our students."

The grade school maxim to "show your work" doesn't hold in the average introductory statistics class, said Mine Cetinkaya-Rundel, assistant professor of the practice in the Duke statistics department. In a typical workflow, a college-level statistics student will perform data analysis in one software package, but transfer the results into something better suited to presentation, like Microsoft Word or Microsoft PowerPoint.

Though standard, this workflow divorces the raw data and analysis from the final results, making it difficult for students to retrace their steps. The process can give rise to errors, and in many cases, the authors write, "the copy-and-paste paradigm enables, and even encourages, selective reporting."

"Usually, a data analysis report, even a published paper, isn't going to include the code," Cetinkaya-Rundel said. "But at the intro level, where this is the first time students are exposed to this workflow, it helps to keep intact both the final results and the code used to generate them."

Enter R Markdown, a statistical package that integrates seamlessly with the programming language R. The team chose R Markdown for its ease of use -- students wouldn't have to learn a new computer syntax -- and because it combines the raw data, computing and written analysis into one HTML document. The researchers hoped a single HTML file would give students a start-to-finish understanding of assignments, as well as make studying and grading easier.

The study introduced R Markdown to 417 introductory statistics students (272 from Duke University, 145 from Smith College) during the 2012-2013 school year. Instructors emphasized the lesson of reproducibility throughout each course and surveyed 70 students about their experience using R Markdown for homework assignments.

The survey, conducted once at the beginning of the semester and once at the end, showed gradual gains in student preference for R Markdown. The percentage of respondents who indicated they found R Markdown to be frustrating at first but eventually got the hang of it jumped from 51 to 75 percent. The students vastly preferred it to the alternative, with 70 percent strongly disagreeing that they'd rather use the copy-and-paste method.

The research team also found that even when students had no prior computing experience or expressed negative attitudes toward R Markdown, their grades did not appear to suffer. Future surveys will ask more pointed questions about how much of the lesson on reproducibility students absorb from the modified curricula.

As the use and analysis of big data becomes increasingly sophisticated, the team writes, the ability of researchers to retrace steps and achieve the same statistical outcomes will only grow in significance.

Other authors on the paper included Andrew Bray and Linda Loi of Smith College and Nicholas Horton of Amherst College.

This researched received partial support from Project MOSAIC and the National Science Foundation (grant DUE-0920350).


Story Source:

The above story is based on materials provided by Duke University. Note: Materials may be edited for content and length.


Journal Reference:

  1. Baumer, Ben; Cetinkaya-Rundel, Mine; Bray, Andrew; Loi, Linda; Horton, Nicholas J. R Markdown: Integrating A Reproducible Analysis Tool into Introductory Statistics. Technological Innovations in Statistics Education, 2014 [link]

Cite This Page:

Duke University. "To teach scientific reproducibility, start young." ScienceDaily. ScienceDaily, 28 February 2014. <www.sciencedaily.com/releases/2014/02/140228140138.htm>.
Duke University. (2014, February 28). To teach scientific reproducibility, start young. ScienceDaily. Retrieved July 23, 2014 from www.sciencedaily.com/releases/2014/02/140228140138.htm
Duke University. "To teach scientific reproducibility, start young." ScienceDaily. www.sciencedaily.com/releases/2014/02/140228140138.htm (accessed July 23, 2014).

Share This




More Science & Society News

Wednesday, July 23, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Boeing Ups Outlook on 52% Profit Jump

Boeing Ups Outlook on 52% Profit Jump

Reuters - Business Video Online (July 23, 2014) Commercial aircraft deliveries rose seven percent at Boeing, prompting the aerospace company to boost full-year profit guidance- though quarterly revenues missed analyst estimates. Bobbi Rebell reports. Video provided by Reuters
Powered by NewsLook.com
Thousands Who Can't Afford Medical Care Flock to Free US Clinic

Thousands Who Can't Afford Medical Care Flock to Free US Clinic

AFP (July 23, 2014) America may be the world’s richest country, but in terms of healthcare, the World Health Organisation ranks it 37th. Thousands turned out for a free clinic run by "Remote Area Medical" with a visit from the Governor of Virginia. Duration: 2:40 Video provided by AFP
Powered by NewsLook.com
Six Indicted in StubHub Hacking Scheme

Six Indicted in StubHub Hacking Scheme

AP (July 23, 2014) Six people were indicted Wednesday in an international ring that took over more than 1,000 StubHub users' accounts and fraudulently bought tickets that were then resold. (July 23) Video provided by AP
Powered by NewsLook.com
9/11 Commission Members Warn of Terror "fatigue" Among American Public

9/11 Commission Members Warn of Terror "fatigue" Among American Public

Reuters - US Online Video (July 22, 2014) Ten years after releasing its initial report, members of the 9/11 Commission warn of the "waning sense of urgency" in combating terrorists attacks. Mana Rabiee reports. Video provided by Reuters
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:
from the past week

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins