7.3.6.5.1 Meta-analysis of observational research

A meta-analysis is a statistical procedure that combines the findings from multiple primary studies into a single overall summary estimate. A meta-analysis can be conducted to improve statistical power to detect a treatment effect, to estimate a summary average effect, to identify sub-groups associated with a negative outcome or a beneficial effect, and to explore differences in the size or direction of the treatment effect associated with study-specific variables. Interpretation of summary effect sizes from meta-analyses of epidemiological studies addressing etiological issues is difficult because of the differences in the factors controlled for in multivariate analyses from individual studies, and also because of poor reporting in the original studies with lack of adequate or complete details. For more information and guidance on meta-analysis, refer to Chapter 3 of this manual. 

An overall effect size is reported in a meta-analysis. It is computed for each study and the findings are pooled together to draw overall inferences. There are many different types of effect size and it is possible to convert one effect size into another, so each really just offers a differently scaled measure of the strength of an effect or a relationship. Reviewers should be aware that there are different guidelines for the interpretation of practical significance of the effect sizes such as ORs and RRs (Tufanaru C, Huang WJ et al. 2012). One proposed guide for interpretation of effect sizes suggests that a value of 2 for a risk estimate (such as a relative risk RR or an odds ratio OR) is considered the minimum significant value from a practical point of view; a value of 3 is considered moderate significant; a value of 4 is considered to indicate strong significance from a practical point of view (Tufanaru C, Huang WJ et al. 2012).

Frequently primary published studies investigating risk of an exposure will design the study and present the available data at different levels of the exposure, or in different categories to reflect a ‘dose-response’ relationship between the exposure and outcome variable. Difficulties will naturally arise if different studies have used different exposure categories and have presented this data in a variety of different ways. A dose response relationship between an exposure and the outcome is most commonly investigated to strengthen the support for causal inference or causation (Greenland and Longnecker 1992, Bekkering, Harris et al. 2008). Individual studies may present results in a stratified manner, either across different exposure groups or in different quantiles. For example, considering the risk of alcohol intake and lung cancer, the data may be presented as different exposure groups such as in glasses/week or in grams of alcohol. Irrespective of this, methods are available to combine the results of individual studies presenting such ‘trend’ data. Dependent on the type of data presented from such a dose response investigation, accepted methods exist to summarize the data to a consistent risk estimate which can then be subsequently used in meta-analysis.

Bekkering et al in a study on the usability of results in a meta-analysis reported that majority of usable results reported were odds, risk, or hazard ratios that compared one or more exposure categories with a baseline category (Bekkering, Harris et al. 2008). They further suggest some advantages in reporting results in ORs, RRs and HRs, which include checking informally for nonlinear exposure effects, and easier interpretation of the magnitude of the association (Bekkering, Harris et al. 2008). In case of nonlinear associations, there is a risk for conclusions from dose-response meta-analysis being misleading and it is suggested that linearity assumptions be checked for each study, when conducting dose-response meta-analysis (Greenland and Longnecker 1992, Bekkering, Harris et al. 2008). Bekkering et al, Chene and Thompson, Greenland and Longnecker, Hamling et al, and Orsini et al describe methods for conducting linear and non-linear dose-response meta-analyses. Essentially, for linear dose-response meta-analysis, the method involves estimation of a linear dose-response curve for each study when combining studies with different exposure category definitions. Further, it requires the numbers of cases and noncases (outcomes) and persons/person-years (person-time) and the effect estimates (RR or OR) with confidence intervals for at least three quantitative exposure categories (Aromataris, Hopp L et al. 2011).

A note on heterogeneity (refer to Chapter 3 for more details)

Despite the impediment to meta-analysis that heterogeneity of the published data presents, be it for methodological, clinical or statistical reasons, meta-analysis of observational studies to inform etiology and risk is almost always possible and can offer a valid means to explore heterogeneity and its impact within a data set. A combined analysis of individual studies, beyond the outright aim of increased precision due to increased sample size, may be desirable as it allows the exploration of potential confounders and interactions and other modifying effects that may explain the heterogeneity among the included studies. It is suggested that the decision to conduct meta-analysis should not be just based on statistical considerations regarding heterogeneity but should be based on the review question, the characteristics of the studies, and the interpretability of the results.