Featured Research

from universities, journals, and other organizations

Call for better social science research transparency

Date:
January 2, 2014
Source:
University of California - Berkeley
Summary:
A group of social scientists from across the United States say it is time for more stringent and transparent standards in social science research to improve the work's substance, impact and credibility.

In the January 3, 2014 edition of the journal Science, an interdisciplinary group is calling on scholars, funders, journal editors and reviewers to adopt more stringent and transparent standards to give social science research more credibility, substance and impact.

The authors, led by a University of California, Berkeley, economist, hope to change a set of practices that they contend has contributed to a distorted body of research that tends to exaggerate the effectiveness of programs that deal with important issues affecting millions of people including health, agriculture, education and environmental policy.

They cite as an example of flawed research a 2010 paper by Harvard University economists Carmen Reinhart and Kenneth Rogoff. It concluded that that when gross external debt hits 60 percent of the Gross Domestic Product, a country's annual growth declines 2 percent, and when debt exceeds 90 percent, growth is roughly cut in half. Budget hawks seized on its conclusions until other economists reviewing the work found coding errors, selective exclusion of available data, and unconventional weighting of summary statistics.

To prevent such errors from working their way into public policy making the 19 authors of the Science paper recommend key changes that include:

  • Documenting and disclosing primary information about data collection and analysis.
  • Preparing and registering pre-analysis plans to distinguish hypothesis testing from exploratory research. These written plans provide a step-by-step account of how a researcher will analyze data before they have seen the data.
  • Archiving and sharing research materials, plans and data through open channels that enable independent researchers to test and extend reported results. The materials can be made public when research is completed.

"Even if we believe that we already have enough self-correction in place, our research will be more credible if we institute these three approaches. We need to raise our game," said UC Berkeley economist Edward Miguel, the lead author of the Science article, "Promoting Transparency in Social Science Research."

"Publish or Perish"

The authors largely blame a "publish or perish" reward structure for academics, saying the odds of getting a study paper published improve if a scholar can promise novel, "theoretically tidy" or statistically significant results rather than more nuanced, mixed or perplexing findings. They also blame scholarly journals and funders for lax oversight of mistakes and confusing information.

"The program suggested by our article would not fundamentally change the nature of social science," said David Laitin, a political science professor at Stanford University and a co-author of the article. "Rather, it would have small implications in the way we collect and report our data. It would also invite us to put more attention to the replication of well-known findings rather than investigations of new factors. But these small changes in our practices could have large effects in revising what we thought were well-established findings."

Miguel said the push toward greater social science research transparency also has been influenced by increasing use of better research designs, new "big data" software tools, growing interest by governments and advocacy groups for evidence-based policy making, and public insistence on greater transparency.

Changes underway

A few institutions are already taking similar steps to those proposed in the Science article. The American Political Science Association adopted guidelines in 2012 making it an ethical obligation for researchers to back up their claims by making their data accessible and being clear about how they produce their results. Related measures have been taken by several psychological journals, the U.S. Office of Management and Budget, and the American Economic Association, with its design registry of randomized trials.

Meanwhile, the Center for Open Science has established an online collaboration tool called the Open Science Framework that enables research teams to easily register their hypotheses and pre-analysis plans, and to make public their data.

Several of the Science authors are members of the Berkeley Initiative for Transparency in the Social Sciences (BITSS), a network that launched a year ago at UC Berkeley to promote discussions and on transparency.

"Discover and disseminate the truth"

BITSS members and Science co-authors Leif Nelson of UC Berkeley's Haas School of Business, and Joseph Simmons and Uri Simonsohn of the Wharton School of the University of Pennsylvania wrote a related 2011 paper, "False-positive psychology: Undisclosed flexibility in data collection and analysis allow presenting anything as significant."

"Our goal as scientists is not to publish as many articles as we can, but to discover and disseminate truth. Many of us -- and this includes the three authors of this article -- often lose sight of this goal, yielding to the pressure to do whatever is justifiable to compile a set of studies that we can publish," they wrote, adding that too often researchers convince themselves that the most publishable outcome must be the best.

The Science article authors maintain that rather than stifle creativity or create undue burdens for researchers, better transparency practices will improve the scientific integrity and impact of their work.

"An open science is a more credible science," said Brian Nosek, a professor of psychology at the University of Virginia, a founder of the Open Science Framework and another of the Science article co-authors.


Story Source:

The above story is based on materials provided by University of California - Berkeley. The original article was written by Kathleen Maclay. Note: Materials may be edited for content and length.


Journal Reference:

  1. E. Miguel, C. Camerer, K. Casey, J. Cohen, K. M. Esterling, A. Gerber, R. Glennerster, D. P. Green, M. Humphreys, G. Imbens, D. Laitin, T. Madon, L. Nelson, B. A. Nosek, M. Petersen, R. Sedlmayr, J. P. Simmons, U. Simonsohn, and M. Van Der Laan. Promoting Transparency in Social Science Research. Science, January 2014

Cite This Page:

University of California - Berkeley. "Call for better social science research transparency." ScienceDaily. ScienceDaily, 2 January 2014. <www.sciencedaily.com/releases/2014/01/140102142002.htm>.
University of California - Berkeley. (2014, January 2). Call for better social science research transparency. ScienceDaily. Retrieved September 1, 2014 from www.sciencedaily.com/releases/2014/01/140102142002.htm
University of California - Berkeley. "Call for better social science research transparency." ScienceDaily. www.sciencedaily.com/releases/2014/01/140102142002.htm (accessed September 1, 2014).

Share This




More Science & Society News

Monday, September 1, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Get on Your Bike! London Cycling Popularity Soars Despite Danger

Get on Your Bike! London Cycling Popularity Soars Despite Danger

AFP (Sep. 1, 2014) Wedged between buses, lorries and cars, cycling in London isn't for the faint hearted. Nevertheless the number of people choosing to bike in the British capital has doubled over the past 15 years. Duration: 02:27 Video provided by AFP
Powered by NewsLook.com
Thailand Totters Towards Waste Crisis

Thailand Totters Towards Waste Crisis

AFP (Sep. 1, 2014) Fears are mounting in Bangkok that poor planning and lax law enforcement are tipping Thailand towards a waste crisis. Duration: 01:21 Video provided by AFP
Powered by NewsLook.com
California Passes 'yes-Means-Yes' Campus Sexual Assault Bill

California Passes 'yes-Means-Yes' Campus Sexual Assault Bill

Reuters - US Online Video (Aug. 30, 2014) California lawmakers pass a bill requiring universities to adopt "affirmative consent" language in their definitions of consensual sex, part of a nationwide drive to curb sexual assault on campuses. Linda So reports. Video provided by Reuters
Powered by NewsLook.com
As Drought Continues LA "water Police" Fight Waste

As Drought Continues LA "water Police" Fight Waste

AFP (Aug. 29, 2014) In the midst of a historic drought, Los Angeles is increasing efforts to go after people who waste water. Five water conservation "cops" drive around the city every day educating homeowners about the drought. Duration: 02:17 Video provided by AFP
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:
from the past week

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins