The JBI methodology for mixed methods systematic reviews aligns with the typology developed by Hong et al. 2017. That is, that the review approach can either be convergent (where the synthesis occurs simultaneously) or sequential (where the synthesis occurs consecutively). However, based on minimal usage of the sequential approach, this guidance for JBI mixed methods systematic reviews currently focuses exclusively on the convergent approach. The convergent design can be broken down into a series of methods that have been simplified into two groups – convergent integrated (involves data transformation that allows reviewers to combine quantitative and qualitative data) and convergent segregated (involves independent synthesis of quantitative data and qualitative data leading to the generation of quantitative evidence and qualitative evidence which are then integrated together). The nature/type of question/s that is/are posed in the systematic review dictates the approach the reviewer should follow for the synthesis.

Nature of the question

The reviewer needs to consider if the review question can be addressed by both quantitative and qualitative studies or if the focus of the review is on different aspects or dimensions of a particular phenomenon of interest. Here are two scenarios highlighting the different question(s) a reviewer may pose for a mixed methods systematic review.

Scenario 1

Scenario 2

Following question development, the steps involved in quantitative and qualitative systematic reviews apply to mixed methods systematic reviews, such as development of eligibility criteria, literature searching and retrieval, critical appraisal and data extraction (please see Chapter 2 and Chapter 3 of the JBI Manual for Evidence Synthesis for further information). Hence, the guidance described in this section will focus on synthesis and the distinct features of a mixed method systematic review – that is, the integration of quantitative and qualitative evidence, and the transformation of quantitative and qualitative data. Ultimately which approach is utilized will depend on the nature of the question(s) posed, as outlined above.

Approaches to synthesis and integration

If the review question can be addressed by both quantitative and qualitative research designs the convergent integrated approach should be followed; if the focus of the review is on different aspects or dimensions of a particular phenomenon of interest the convergent segregated approach is undertaken. Let’s now take another look at our two examples to explain why.

Scenario 1

 Scenario 2

The convergent integrated approach, suggested for Scenario 1 above, refers to a process of combining extracted data from quantitative studies (including data from the quantitative component of mixed methods studies) and qualitative studies (including data from the qualitative component of mixed methods studies), and involves data transformation. It is recommended that quantitative data be ‘qualitized’, as codifying quantitative data is less error-prone than attributing numerical values to qualitative data (The Joanna Briggs Institute, 2014). ‘Qualitizing’ involves extracting data from quantitative studies and translating or converting it into ‘textual descriptions’ to allow integration with qualitative data. ‘Qualitizing’ involves a narrative interpretation of the quantitative results.

At the simplest level, qualitized data might comprise describing a sample (or members of it) using word categories based on supplementary descriptive statistics such as average or percentage scores (Bazeley, 2012). The study by Cohen et al. 2003 (part of the review by Holley et al. 2017 outlined in Scenario 1 above) aimed to examine the perceptions of adolescents with asthma and their attitudes towards self-treatment. Qualitization identified: 29% of survey participants reported feeling embarrassed having an asthma attack while with friends (Cohen et al., 2003). Qualitized data can also include profiling of the sample using cluster or factor analysis (Bazeley, 2012). Data with a temporal or longitudinal component (Bazeley, 2012), or those that examine associations and relationships using inferential statistics such as linear or logistic regression analysis also have narrative potential and can therefore be qualitized by identifying variables included in the analysis. For example the study by Kyngäs (2000) (also in Holley et al., 2017) identified factors that predict compliance with health regimens by adolescents with asthma using logistic regression. Transformation identified: support from nurses as a significant factor in predicting compliance with health regimens by adolescents with asthma (OR = 56.87, 95% 17.15-88.58).  By qualitizing, the reviewer converts the ‘quantities’ into declarative stand-alone sentences, in a way that answers the review question.

These textual descriptions are then assembled and pooled with the qualitative data extracted directly from qualitative studies. Similar to the meta-aggregative approach for JBI qualitative reviews, reviewers are required to then undertake repeated, detailed examination of the assembled data to identify categories on the basis of similarity in meaning. A category will integrate two or more: qualitative data, ‘qualitized’ data or a combination of both. In some instances however, data may not have the same meaning as others and therefore cannot be combined to form a category. Where possible, categories are then aggregated to produce the overall finding(s) of the review. This process is illustrated in Figure 8.1.

Open

Figure 8.1: Convergent Integrated Approach. Qualitized findings are assembled into categories with qualitative findings extricated directly from qualitative studies based on similarity of meaning.

Using the example outlined above (Scenario 1), reviewers were able to determine six key barriers and/or enablers regarding self-management of asthma, which related to knowledge, lifestyle, beliefs and attitudes, relationships, intrapersonal characteristics and communication (Holley et al., 2017).  

The convergent segregated approach consists of conducting a separate quantitative synthesis and qualitative synthesis, followed by integration of the results derived from each of the syntheses. By integrating the quantitative and qualitative synthesized findings, we are able to have a greater depth of understanding of the phenomena of interest compared to undertaking two separate component syntheses without formally linking the two sets of evidence. In Scenario 2 above, quantitative data is synthesized in the form of a meta-analysis (or a narrative summary if meta-analysis is not possible) to determine the effects of mindfulness-based interventions on nurses. Additionally, all the qualitative data is pooled (in the case of the JBI approach, through the process of meta-aggregation or narrative summary if deemed inappropriate – refer to Chapter 2 of the JBI Manual for Evidence Synthesis for further information) to determine the experiences/perceptions of nurses receiving these interventions. There is no order to which synthesis is done first as they are independent; however, both must be completed before moving onto the next step: integration of quantitative evidence and qualitative evidence. This next step involves juxtaposing the synthesized quantitative results with the synthesized qualitative findings and organizing or linking the results and findings into a line or argument to produce an overall ‘configured analysis’. This is where the reviewer considers how (and if) the results and findings complement each other by using one type of evidence to explore, contextualize or explain the findings of the other type of evidence. In this step, results and findings cannot be reduced but are organized into a coherent whole (Sandelowski, Voils & Barroso, 2006). In this approach, the reviewer repeatedly compares the results of the quantitative synthesis with the findings of the qualitative synthesis, analyzing the intervention which had been investigated for effectiveness (quantitative) in light of the experiences of the participants (qualitative). The following questions act as a guide for this process:

• Are the results/findings from individual synthesis supportive or contradictory?

• Does the qualitative evidence explain why the intervention is or is not effective?

• Does the qualitative evidence help explain differences in the direction and size of effect across the included quantitative studies?

• Which aspects of the quantitative evidence are/are not explored in the qualitative studies?

• Which aspects of the qualitative evidence are/are not tested in the quantitative evidence?

In some instances, the reviewer may find that the results of quantitative studies are not complementary or have no relationship with the findings of the qualitative studies, or vice-versa. In some cases the reviewer may identify gaps where further research may be useful to explain the contradictory findings or when there is no relationship between the qualitative findings and quantitative findings.

In Scenario 2 (mindfulness-based interventions for nurses), results from statistical meta-analysis showed significant reductions in anxiety and depression following treatment, whereas the qualitative synthesis highlighted improvements in areas such as well-being and work performance. In this example the qualitative synthesis highlighted factors not considered or covered in the quantitative synthesis which led to stronger support of the intervention as well as recommendations for future research (Guillaumie et al., 2017).

This integration follows a formal, structured process which is reported in the results section of the review (i.e. it “marries” the results of separate syntheses).  The JBI Framework for undertaking a mixed methods systematic review is outlined in Figure 8.2.

Regardless of the approach taken, the ability to undertake a mixed methods synthesis and integration will ultimately depend on the evidence located and subsequently included in the review. As in a quantitative review focussing on a question of effectiveness where the aim is to be able to conduct a meta-analysis (or similarly a meta-aggregation in a qualitative review), in a mixed methods systematic review there may not be sufficient evidence available, the data may be limited in its ‘richness’ or thickness of description or the evidence located may not be similar enough to combine or link together. In these situations, the authors may need to undertake a narrative synthesis instead, much like in a quantitative review when a meta-analysis is not possible. 

The JBI System for the Unified Management, Assessment and Review of Information (JBI SUMARI) supports reviewers to undertake a mixed methods systematic review using both the convergent integrated and the convergent segregated approaches.  

Open

Figure 8.2: The JBI Framework for mixed methods systematic reviews