Featured Research

from universities, journals, and other organizations

Most large treatment effects of medical interventions come from small studies, report finds

Date:
October 23, 2012
Source:
American Medical Association (AMA)
Summary:
In an examination of the characteristics of studies that yield large treatment effects from medical interventions, these studies were more likely to be smaller in size, often with limited evidence, and when additional trials were performed.

In an examination of the characteristics of studies that yield large treatment effects from medical interventions, these studies were more likely to be smaller in size, often with limited evidence, and when additional trials were performed, the effect sizes became typically much smaller, according to a study in the October 24/31 issue of JAMA.

"Most effective interventions in health care confer modest, incremental benefits," according to background information in the article. "Large effects are important to document reliably because in a relative scale they represent potentially the cases in which interventions can have the most impressive effect on health outcomes and because they are more likely to be adopted rapidly and with less evidence. Consequently, it is important to know whether, when observed, very large effects are reliable and in what sort of experimental outcomes they are commonly observed. … Some large treatment effects may represent entirely spurious observations. It is unknown how often studies with seemingly very large effects are repeated."

Tiago V. Pereira, Ph.D., of the Health Technology Assessment Unit, German Hospital Oswaldo Cruz, Sao Paulo, Brazil, and colleagues conducted a study to evaluate the frequency and features of very large treatment effects of medical interventions that are first recorded in a clinical trial. For the study, the researchers used data from the Cochrane Database of Systematic Reviews (CDSR) and assessed the types of treatments and outcomes in trials with very large effects, examined how often large-effect trials were followed up by other trials on the same topic, and how these effects compared against the effects of the respective meta-analyses.

Among 3,545 available reviews, 3,082 contributed usable information on 85,002 forest plots (a graphical display designed to illustrate the relative strength of treatment effects in multiple studies). Overall, 8,239 forest plots (9.7 percent) had a nominally statistically significant very large effect in the first published trial, group A; 5,158 (6.1 percent) had a nominally statistically significant very large effect found only after the first published trial, group B; and 71,605 (84.2 percent) had no trials with significant very large effects, group C. The researchers found that nominally significant very large effects arose mostly from small trials with few events. For the index trials, the median [midpoint] number of events was only 18 in group A and 15 in the group B. The median number of events in the group C index trials was 14.

The authors also observed that 90 percent and 98 percent of the very large effects observed in first and subsequently published trials, respectively, became smaller in meta-analyses that included other trials; the median odds ratio decreased from approximately 12 to 4 for first trials, and from 10 to 2.5 for subsequent trials.

Topics with very large effects were less likely than other topics to address mortality. Across the whole CDSR, there was only 1 intervention with large beneficial effects on mortality and no major concerns about the quality of the evidence (for a trial on extracorporeal oxygenation for severe respiratory failure in newborns).

"… this empirical evaluation suggests that very large effect estimates are encountered commonly in single trials. Conversely, genuine very large effects with extensive support from substantial evidence appear to be rare in medicine and large benefits for mortality are almost entirely nonexistent. As additional evidence accumulates, caution may still be needed, especially if there is repetitive testing of accumulating trials. Patients, clinicians, investigators, regulators, and the industry should consider this in evaluating very large treatment effects when the evidence is still early and weak," the researchers write.

Editorial: Improving the Health of Patients and Populations Requires Humility, Uncertainty, and Collaboration

Andrew D. Oxman, M.D., of the Norwegian Knowledge Centre for the Health Services, Oslo, Norway, comments on the findings of this study in an accompanying editorial.

"Clinicians should be humble about the ability to prevent or treat most health problems, although a large range of effective interventions are available, some with large effects, most have modest (albeit important) effects and the effects of many are uncertain. Acknowledging uncertainty is the first, essential step to reducing important uncertainties through well-designed evaluations. Collaboration is essential to reduce those uncertainties by identifying and agreeing to priorities for evaluation and making sure not to continue to waste scarce resources for research on unimportant questions or poorly designed evaluations. Clinicians need to collaborate on needed evaluations and synthesizing and making the results of evaluations readily available to inform decisions about how best to improve the health of individuals and populations."


Story Source:

The above story is based on materials provided by American Medical Association (AMA). Note: Materials may be edited for content and length.


Journal References:

  1. Pereira TV, Horwitz RI, Ioannidis JA. Empirical Evaluation of Very Large Treatment Effects of Medical Interventions. JAMA, 2012; 308 (16): 1676-1684 DOI: 10.1001/jama.2012.13444
  2. Oxman AD. Improving the Health of Patients and Populations Requires Humility, Uncertainty, and Collaboration. JAMA, 2012; 308 (16): 1691-1692 DOI: 10.1001/jama.2012.14477

Cite This Page:

American Medical Association (AMA). "Most large treatment effects of medical interventions come from small studies, report finds." ScienceDaily. ScienceDaily, 23 October 2012. <www.sciencedaily.com/releases/2012/10/121023161103.htm>.
American Medical Association (AMA). (2012, October 23). Most large treatment effects of medical interventions come from small studies, report finds. ScienceDaily. Retrieved September 23, 2014 from www.sciencedaily.com/releases/2012/10/121023161103.htm
American Medical Association (AMA). "Most large treatment effects of medical interventions come from small studies, report finds." ScienceDaily. www.sciencedaily.com/releases/2012/10/121023161103.htm (accessed September 23, 2014).

Share This



More Health & Medicine News

Tuesday, September 23, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Ebola Costs Keep Mounting

Ebola Costs Keep Mounting

Reuters - Business Video Online (Sep. 23, 2014) The WHO has warned up to 20,000 people could be infected with Ebola over the next few weeks. As Sonia Legg reports, the implications for the West African countries suffering from the disease are huge. Video provided by Reuters
Powered by NewsLook.com
Ebola Cases Could Reach 1.4 Million Within 4 Months

Ebola Cases Could Reach 1.4 Million Within 4 Months

Newsy (Sep. 23, 2014) Health officials warn that without further intervention, the number of Ebola cases in Liberia and Sierra Leone could reach 1.4 million by January. Video provided by Newsy
Powered by NewsLook.com
WHO: Ebola Cases to Triple in Weeks Without Drastic Action

WHO: Ebola Cases to Triple in Weeks Without Drastic Action

AFP (Sep. 23, 2014) The number of Ebola infections will triple to 20,000 by November, soaring by thousands every week if efforts to stop the outbreak are not stepped up radically, the WHO warned in a study on Tuesday. Duration: 01:01 Video provided by AFP
Powered by NewsLook.com
5 Ways Men Can Prevent Most Heart Attacks

5 Ways Men Can Prevent Most Heart Attacks

Newsy (Sep. 23, 2014) No surprise here: A recent study says men can reduce their risk of heart attack by maintaining a healthy lifestyle, which includes daily exercise. Video provided by Newsy
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:

Strange & Offbeat Stories


Health & Medicine

Mind & Brain

Living & Well

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins