Early results from clinical trials not all they're cracked up to be, shows new research

"This phenomenon of exaggerated early results was present in a whopping 37 percent of the studies we reviewed," says Fares Alahdab, M.D., lead author of the study and a research fellow in Mayo Clinic's Evidence-Based Practice Center. "Physicians and patients should be cautious about new or early clinical trial evidence. Exaggerated results could lead to false hope as well as possibly harmful effects."

Dr. Alahdab and the team of researchers led by the center's director, M. Hassan Murad, M.D., reviewed thousands of research articles from the top 10 general medical journals using impact factors. (Impact factors are an internationally accepted ranking system for scientific journals). They collected 70 meta-analysis articles published during an 8½-year period (Jan. 1, 2007, to June 23, 2015) that included the results of 930 clinical trials.

They discovered that the first or second studies (in terms of publication year) reported an effect that was 2.67 times larger than what eventually was shown when subsequent trials were published. The team focused its research on clinical trials evaluating drugs or devices for use in treating chronic conditions -- those conditions that generally persist over time, including cancer, stroke, heart disease, diabetes and kidney disease.

"Often, patients are living with more than one chronic condition, and they and their doctors watch for research about new treatments," says Dr. Murad. "They need to be aware that the effect seen in earlier trials may not bear out over time and may be much more modest."

"Some people may think this is an anti-innovation message," Dr. Murad says. "To the contrary, we welcome new treatments. We just want people to know that the benefit seen in real practice, when treatments are given to people with various comorbidities and in different settings, may be smaller than what was seen in the earliest clinical trials."

When publishing early clinical trial results, the research team also hopes their research colleagues will consider the possibility of the Proteus effect -- a name often given to the phenomenon of early exaggeration followed by moderation over time. The research team hopes their colleagues will temper their language to avoid providing false optimism.