Featured Research

from universities, journals, and other organizations

Methods proposed to improve how observational studies are conducted

Date:
August 25, 2011
Source:
National Institute of Statistical Sciences
Summary:
Statisticians point out that medical and other observational studies often produce results that are later shown to be incorrect. They now suggest ways to fix the system.

S. Stanley Young, assistant director for bioinformatics at the National Institute of Statistical Sciences (NISS), and Alan Karr, director at NISS, have published a non-technical article in the September issue of Significance magazine pointing out that medical and other observational studies often produce results that are later shown to be incorrect, and -- invoking a quality control perspective -- suggest ways to fix the system.

Related Articles


Their central point is that the current system of publication in peer-reviewed journals relies on post-production inspection to ensure quality, a practice that has disappeared from modern industry in favor of controlling the process instead: quality control is now process control, not product control. They cite W. Edwards Deming, considered by many the most innovative thinker ever about quality, arguing not only for process control, but also that the problem lies with the managers -- funders and journals -- rather with than the workers -- individual researchers who respond rationally to the current set of incentives.

Young and Karr describe both their and others' studies of the extent to which observational studies do not replicate. Published claims such as "coffee causes pancreatic cancer," or "women eating breakfast cereal are more likely to have boy babies," have been refuted by subsequent studies and analyses. When these studies reach the popular media and influence individual consumers, the burden falls not just on science but also on society. And even if there were no impact on the public, scarce research resources, both money and personnel, have been squandered.

The paper describes several technical difficulties with observational studies, among them multiple testing (if enough questions are asked, some will yield false positive answers), bias (systematic error) and multiple modeling (searching among mathematical models until one is found that "fits the data"). Publication bias is another issue: papers reporting positive scientific results (for example, an association between Type A personalities and heart attacks) are more likely to be published than those reporting negative results, even though the latter may be as important scientifically.

Young and Karr recommend that when a study is submitted for publication, the data be split into two sets, a modeling data set and a holdout data set. Journals would then accept or reject papers based on the analysis of the modeling data set without knowing the results of applying the methods to the holdout set. But then the journal would also publish an addendum to the paper giving the results of the analysis of the holdout set.

Significance magazine is published by the Royal Statistical Society of the UK and the American Statistical Society.


Story Source:

The above story is based on materials provided by National Institute of Statistical Sciences. Note: Materials may be edited for content and length.


Cite This Page:

National Institute of Statistical Sciences. "Methods proposed to improve how observational studies are conducted." ScienceDaily. ScienceDaily, 25 August 2011. <www.sciencedaily.com/releases/2011/08/110825123832.htm>.
National Institute of Statistical Sciences. (2011, August 25). Methods proposed to improve how observational studies are conducted. ScienceDaily. Retrieved October 24, 2014 from www.sciencedaily.com/releases/2011/08/110825123832.htm
National Institute of Statistical Sciences. "Methods proposed to improve how observational studies are conducted." ScienceDaily. www.sciencedaily.com/releases/2011/08/110825123832.htm (accessed October 24, 2014).

Share This



More Computers & Math News

Friday, October 24, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Microsoft Riding High On Strong Surface, Cloud Performance

Microsoft Riding High On Strong Surface, Cloud Performance

Newsy (Oct. 24, 2014) — Microsoft's Q3 earnings showed its tablets and cloud services are really hitting their stride. Video provided by Newsy
Powered by NewsLook.com
The Best Apps to Organize Your Life

The Best Apps to Organize Your Life

Buzz60 (Oct. 23, 2014) — Need help organizing your bills, schedules and other things? Ko Im (@konakafe) has the best apps to help you stay on top of it all! Video provided by Buzz60
Powered by NewsLook.com
Nike And Apple Team Up To Create Wearable ... Something

Nike And Apple Team Up To Create Wearable ... Something

Newsy (Oct. 23, 2014) — For those looking for wearable tech that's significantly less nerdy than Google Glass, Nike CEO Mark Parker says don't worry, It's on the way. Video provided by Newsy
Powered by NewsLook.com
Chameleon Camouflage to Give Tanks Cloaking Capabilities

Chameleon Camouflage to Give Tanks Cloaking Capabilities

Reuters - Innovations Video Online (Oct. 22, 2014) — Inspired by the way a chameleon changes its colour to disguise itself; scientists in Poland want to replace traditional camouflage paint with thousands of electrochromic plates that will continuously change colour to blend with its surroundings. The first PL-01 concept tank prototype will be tested within a few years, with scientists predicting that a similar technology could even be woven into the fabric of a soldiers' clothing making them virtually invisible to the naked eye. Matthew Stock reports. Video provided by Reuters
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
 
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:  

Breaking News:

Strange & Offbeat Stories

 

Space & Time

Matter & Energy

Computers & Math

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:  

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile iPhone Android Web
Follow Facebook Twitter Google+
Subscribe RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins