The universal steps involved in a systematic review (e.g. formulation of review question/s, establishing eligibility criteria, development of a search strategy, searching and retrieval of relevant studies, critical appraisal of included studies, data extraction, and synthesis) also apply to a MMSR. However, unique aspects regarding how data is combined and the overall integration of the evidence are additional factors that need to be considered.

To avoid confusion in describing a MMSR, it is important to firstly outline a number of core concepts related to this type of systematic review (Table 8.1). A review of the literature conducted by the authors informed the development of core concepts and the subsequent JBI MMSR approach that is detailed in Section 8.3.

Table 8.1: Summary of concepts related to MMSR

A systematic review examining the different methods available to synthesize quantitative and qualitative data or integrate quantitative and qualitative evidence was undertaken by Hong et al 2017. The review included 459 reviews utilizing a number of different frameworks for integration; however, it identified two predominant frameworks to MMSR: the convergent design (where syntheses occur at the same time) and the sequential design (where syntheses occur one after another). The two frameworks identified in Hong et al.’s (2017) review concur with the seminal work undertaken by Sandelowski and colleagues (2006) who developed three basic designs for MMSR which were adapted from the primary mixed methods literature. They include the: (1) integrated design, (2) segregated design and (3) contingent design (Sandelowski et al., 2006).

1. The integrated design involves integration of transformed data referred to as direct assimilation, which rests on the assumption that quantitative and qualitative data can both address the same research question. As such they can be combined once data have been transformed in the same format (i.e. ‘quantitized’ or ‘qualitized’) (Sandelowski et al., 2006).

2. The segregated design involves integration of evidence through a method referred to as configuration, which is the arrangement of complementary evidence into a line of argument. Complementarity rests on the assumption that quantitative and qualitative evidence address different research questions that are related to the same phenomenon of interest. In other words, quantitative and qualitative evidence address different aspects or dimensions of a phenomenon of interest and therefore they can neither confirm nor refute each other but rather only complement each other. As such, the quantitative evidence and qualitative evidence cannot be directly combined and can only be organized into a coherent whole (Sandelowski et al., 2006).  

3. The contingent design takes a cyclic approach in which synthesis is conducted in order to answer questions raised from the previous synthesis i.e. the results of each synthesis determines the next question to undertake until a comprehensive research synthesis that addresses the reviewers objectives is complete (Sandelowski et al., 2006).  Table 8.2 provides a comparison of these frameworks.

Table 8.2: Comparison of frameworks identified by Hong et al. (2017) and Sandelowski et al. (2006).

The three main considerations in undertaking an MMSR relate to:

1. the sequence in which the synthesis occurs,

2. how data is transformed, and

3. how transformed data or quantitative and qualitative evidence are integrated together.

Sequence of synthesis

As described above, the order of synthesis can be either convergent or sequential. The convergent design is the dominant approach used in MMSR (95% of reviews), with the sequential design only applied in a very small proportion of reviews (5%) (Hong et al., 2017). Consequently, this current MMSR guidance will focus exclusively on convergent approaches.

In the convergent approach the synthesis occurs simultaneously. This can occur at two different stages within the review; dependent on the type of convergent design utilized. In the first instance, synthesis occurs at the data level when quantitative, qualitative and mixed methods studies are extracted concurrently, data is transformed and then analyzed in a parallel manner.

In the second instance, quantitative evidence (from quantitative studies and data from the quantitative component of mixed methods studies) is synthesized separately as is qualitative evidence (from qualitative studies and data from the qualitative component of mixed methods studies) which are then integrated together.

Data transformation

In order for qualitative and quantitative data to be integrated and fully inform the topic, one approach is for the data to be transformed into a mutually compatible format (Voils et al., 2009). Data transformation can occur either by converting qualitative data into quantitative data (i.e. quantitizing) or by converting quantitative data into qualitative data (i.e. qualitizing). Quantitizing is a process in which qualitative data are assigned numerical values. Approaches described in the literature include content analysis, Bayesian analysis and Boolean analysis (Frantzen & Fetters, 2016). Qualitizing refers to quantitative data being converted into themes, categories, typologies or narratives (Frantzen & Fetters, 2016; Heyvaert et al., 2013; Sandelowski et al., 2006). This can be undertaken by thematic analysis, critical interpretative synthesis, meta-narrative synthesis and realist synthesis (Frantzen & Fetters, 2016). Both quantizing and qualitizing approaches are accepted in the literature; however, one is not recommended over the other with both having their strengths and weaknesses.

Integration of findings

Integration refers to how transformed data are merged or how quantitative and qualitative evidence are combined. The literature indicates there are various methods for undertaking integration; some of these are described below

A.     Integration following data transformation

Quantitative approach: this type of integration is applied when qualitative data are quantitized. Commonly used approaches include content analysis and vote counting.

• In content analysis, themes or categories are developed a priori (i.e. before integration) and then all extracted data (i.e. quantitative data and quantitized qualitative data) are coded according to these categories or themes (Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005; Spilsbury et al., 2008). This is followed by creating tabulations of frequency counts to identify key findings (Dixon-Woods et al., 2005; Spilsbury et al., 2008).

• Vote counting involves two steps: first, the findings of the included studies are classified into those that yield positive results, those that yield negative results, and those that show no difference (i.e. not positive and not negative); second, the number of primary studies allocated to each classification are counted (Hayvaert et al., 2017; Hong et al., 2017). The classification which has the most number of counts is declared the ‘winning category’ and therefore provides the most convincing evidence according to the vote-counting approach (Hayvaert et al., 2017; Hong et al., 2017).

Qualitative approach: this type of integration is applied when quantitative data are qualitized; to date, the most common approach to such integration is thematic synthesis. In thematic synthesis, extracted data are coded, followed by grouping of codes which then make up a specific theme (Thomas & Harden, 2008). The descriptive themes might then lead to a conceptual framework. In some instances, a theoretical or conceptual framework is used to develop a priori set of themes on which to organize the codes identified from the analysis of extracted data.

B.    Integration following quantitative and qualitative synthesis

Methods that are often used for integrating a quantitative evidence synthesis with a qualitative evidence synthesis are realist synthesis, narrative summary, thematic synthesis or framework synthesis.

• Realist synthesis is a theory-driven approach aimed at unpacking how an intervention works in a particular context or setting – ‘what works for whom in what circumstances’ (Pawson et al., 2005).

• Narrative summary varies from a ‘simple recounting and description of findings to more reflective accounts that include commentary and higher levels of abstraction to explain complex processes’ (Hayvaert et al., 2017) p.231.

• Thematic synthesis uses coding, groups similar codes and develops descriptive themes to generate an overall summary of findings (Hong et al., 2017; Thomas & Harden, 2008).

• Framework synthesis involves a preliminary identification of themes against which to map and configure the findings from the quantitative and qualitative studies (Carroll et al., 2011).

A summary of the methodological approaches for MMSR is provided in Table 8.3.

Table 8.3:  Summary of methodological approaches for MMSR