4.3 Meta-analysis

Meta-analysis refers to the statistical synthesis of quantitative results from two or more studies. The review protocol should state that statistical meta-analysis of data will be conducted if appropriate and that if meta-analysis is not possible, narrative synthesis will be conducted as the primary mechanism of data synthesis. Narrative summary should be included to supplement the technical details provided on the process and results even if meta-analysis is performed and to provide synthesis of data not captured in statistical meta-analysis.

Meta-analysis should be reserved for the results of studies that are considered similar enough from a clinical and methodological point of view (homogeneous studies). If studies are heterogeneous from a clinical or methodological point of view, then it is uncertain if it is appropriate to synthesize the respective studies into meta-analysis. Any meta-analysis where studies are heterogeneous from a clinical or methodological point of view will require substantial justification by the authors. Clinical heterogeneity refers to differences between studies with regards the participants, interventions, comparators, settings, and outcomes. Methodological heterogeneity refers to the study design and the methodological quality of the studies (risk of bias). Studies that are similar with regards the participants, interventions, comparators, settings, outcomes, study design, and risk of bias may be combined in meta-analysis. The judgement that studies are homogeneous enough and that it is appropriate to combine the studies in meta-analysis should be based on the understanding of the review question, the characteristics of the studies, and the interpretability of the results. The decision should not be based just on statistical considerations regarding heterogeneity (Sutton et al 2000).

The review protocol should specify the appropriate possible, reasonable details regarding the anticipated (pre-planned) meta-analysis:

• Objectives of the meta-analysis,

• Meta-analysis model (fixed effects model or random effects model) and the justification,

• Effect size to be used (OR, RR, etc.),

• Meta-analysis method (Peto method etc.) and justification,

• Statistical testing procedures used for the exploration of statistical heterogeneity (such as Q Cochran test) and the rules used for the interpretation of the results,

• Statistical indicator used for the quantification of statistical heterogeneity (such as I²) and the rules used for the interpretation of the results,

• Pre-planned sensitivity analyses and their justification, and

• Pre-planned subgroup analyses and their justification.