Draft 4.3.10.1 Standard chi-squared test (Cochran test)

The standard chi-squared test (Cochran Q test) for statistical heterogeneity tests the statistical hypothesis that the true treatment effects (the effect size parameters) are the same in all the primary studies included in meta-analysis (Sutton et al 2000). This statistical test uses a test statistic Q that has a chi-squared distribution on k-1 degrees of freedom (k represents the number of studies) under the statistical hypothesis; the corresponding p-value for the test statistic is examined (Sutton et al 2000). The statistical power of the test is in most cases very low due to the small number of studies; heterogeneity may be present even if the Q statistic is not statistically significant at conventional levels of significance such as 0.05. A cut-off significance level of 0.10 rather than the usual 0.05 has been advocated (Sutton et al 2000). If results of the test are statistically significant (p<0.05) the statistical hypothesis that the true treatments effects (the effect size parameters) are the same in all the primary studies included in meta-analysis (the hypothesis of homogeneity) is rejected, therefore, it is considered that there is statistical heterogeneity. With a small number of studies (< 20), the Q test should be interpreted very cautiously (Huedo-Medina et al 2006). It is not appropriate to decide the meta-analysis model (fixed or random effects model) based on the results of the Chi squared statistical test (Q test) for heterogeneity.