How should systematic reviews consider evidence on harms?

There is increasing focus on the importance of using rigorous methods to assess the effectiveness and harms associated with the use of new drugs and other therapies, and recognition of the role of systematic reviews in this process. A systematic review uses predefined, explicit methods to find and appraise all relevant evidence to answer a specific question in healthcare. However, there has been considerable debate as to whether systematic reviews should use evidence from randomized controlled trials or observational studies (or both) in order to collect all the relevant evidence on risk of harms. Some groups have argued that observational studies may produce biased estimates of harm, while randomized trials may be too small to generate useful data on the risk of rare adverse effects.

In the study, Golder and colleagues identified systematic reviews that had compared the risk of specific harms in evidence from randomized controlled trials versus the evidence from observational studies. They found that there was no difference on average in the estimates produced by these two approaches. The authors conclude: "Instead of restricting the analysis to certain study designs, it may be preferable for systematic reviewers of adverse effects to evaluate a broad range of studies that can help build a complete picture of any potential harm and improve the generalisability of the review without loss of validity."

This research was undertaken by Su Golder as part of an MRC fellowship. The views expressed in this presentation are those of the authors and not necessarily those of the MRC.