3.3.12 Extracting findings

The approach and process used to extract findings from the results of the included studies should be presented with enough detail to be readily reproducible. This includes a clear description of how equity considerations were incorporated into the extraction process (e.g. in describing the sample or the context), ensuring that diverse perspectives are accurately represented and biases are minimised.

A standardised data extraction tool should be used for data extraction, and it is recommended that all individuals involved pilot test the tool to ensure understanding, accuracy and consistency. At least two people should independently extract the data, with extraction limited to findings that address the review questions. Any deviations from these recommendations should be documented under limitations.

Data extraction in meta-aggregation is a two-phase process.

Phase 1: Extraction of study characteristics. This phase involves extracting the general details of papers, including the citation details, population, phenomenon of interest and context as well as methodology, methods, setting and details of relevant cultural information retrieved from the studies.

Phase 2: Extraction of findings. A finding is defined as ‘a verbatim extract of the author’s analytic interpretation of their results or ‘data‘ (Lockwood et al. 2015, p. 183). The ‘data’ should be accompanied by one or more illustrations (referred to as an ‘illustration of finding’) from the same source that informs the finding. In some cases, an ‘illustration of finding’ may be inherently linked to multiple findings. For instance, a participant quote might support both a finding about emotional strain and a related finding about coping mechanisms. In such cases, it is appropriate to align the illustration with more than one extracted finding. This approach highlights the interconnections between findings while preserving the richness and depth of the qualitative evidence.

The illustration may include participant voices, fieldwork observations or other forms of data.

In Phase 2, the findings can be extracted at the theme or sub-theme level; this decision lies with the review team. It is important to extract findings that have a clearly articulated and comprehensive meaning or that clearly articulate a concept or process. This can be a guiding principle when deciding to extract at the theme or sub-theme level. Braun and Clarke (2022, p. 229) define a theme as ‘patterns of meaning (e.g. concepts, ideas, experience, sense-making) that are underpinned and unified by a central idea’. It is very common, however, for study authors to present their main themes as ‘topic summaries’ (often just a general label or description of a topic or a range of associated factors), rather than as coherent meaning statements. In this case, it would be more appropriate to extract findings that are presented as meaning statements at sub-theme level. Overall, when deciding whether to extract findings at the theme or sub-theme level, the overarching consideration should be whether the theme or sub-theme expresses a clear and comprehensive pattern of meaning related to the phenomenon of interest. While every effort should be made to extract findings at the same level of detail across studies to ensure consistency, it may not always be feasible. In such instances, it may be necessary to extract findings at different levels between studies to ensure that the meaning is fully captured. This flexibility ensures that the extraction process remains faithful to the underlying concepts or processes conveyed by the study authors.

In cases of missing data, authors can be contacted to clarify and provide data to support the extracted findings. It may be that the data were collected in field notes or that the authors were unable to include an illustration in the paper. If contacting authors for missing data, it is important that the review team avoids re-interpretation of that data.

Important aspects to consider when extracting findings include:

• Processes by which the findings were identified (i.e. repeated reading of text).

• Criteria used to define the findings in the review (e.g. whether the findings were limited to themes and metaphors or if they also included other analytical data from the papers, such as detailed analytical descriptions by the authors, rather than just a thematic analysis).

Assessment of credibility of findings

The concept of credibility revolves around the accurate representation of a finding. Evaluating credibility encompasses various dimensions, such as assessing the ‘goodness of fit’ and representativeness of a given finding (Lincoln & Guba 1985). Goodness of fit refers to the extent to which the findings of a study are trustworthy, believable and reflective of the reality or experiences being investigated (Lincoln & Guba 1985). In qualitative research, achieving goodness of fit is crucial. It involves assessing whether the interpretation and synthesis of findings faithfully capture the nuances, themes and patterns discovered in the original qualitative data. A finding exhibits good fit when it aligns with the depth and richness of the data, ensuring that the finding accurately represents the essence of the participants’ experiences or the phenomenon under investigation (Lincoln & Guba 1985). In a qualitative systematic review, representativeness assesses the comprehensiveness and inclusivity of the selected studies in portraying the diversity of perspectives within the studied phenomenon.

Credibility is auditable and this auditability reflects the commitment to transparency and accountability in research. Credibility is achieved through various means in the study, such as sense-checking with other members of the research team, member checking, careful attention to negative cases (outliers), peer evaluations, secondary analysis by another researcher or direct observation.

As part of a meta-aggregative review, when a finding and its supporting illustration are extracted, a level of credibility is assigned to the finding. This is based on the reviewer’s perception of the degree of support each illustration offers for the specific finding it is associated with. There are three levels of credibility as described below:

• Unequivocal (U): The findings are accompanied by a contextually rooted, detailed, rich and clearly associated illustration.

• Credible (C): The findings are accompanied by an illustration lacking detail, richness or clear association with it.

• Not Supported (NS): The findings are not supported by the data or there are no data to support the finding.

The reviewers should document how the decision was made to allocate these levels and what (if any) issues arose during this process or whether there was good agreement between the review team members. The level of credibility should be presented alongside the extracted finding. Note that unsupported findings are not included in the data synthesis but should be noted in the narrative of the review finding and presented in an appendix.

Examples of findings, illustrations of findings and credibility assessment

Each of the examples below demonstrates the various ways in which a finding can be extracted (themes, metaphors or other analytic data) and the accompanying level of credibility.

Example 1

Finding: Changing as a person (U) (McCloskey et al. 2023).

Illustration of finding: ‘I was overwhelmed with the workload, something I expected, but still the intensity of it was a challenge, especially for a working adult student, with obligations to family, employers and the university. I grew as a person as a result of my experiences in the program.’ (p. 158)

Example 2

Finding: They had friends, family and neighbours that they could call upon, even if they did not require it (U) (Cotton et al. 2024).

Illustration of finding: ‘If I told him that I needed him [son], he would be here in 5 minutes.’ (p. 15)

Example 3

Finding: Monitoring the child’s symptoms and treatments (C) (Min et al. 2022).

Illustration of finding: ‘I didn’t know that he had headaches every day and I didn’t reflect over why he only wanted yoghurt and not real food before we came to specialised dental care.’ (Parent, p. 357)

Example 4

Findings: Moving beyond the constraints of being an EN. (C) (McCloskey et al. 2023).

Illustration of finding: The RNs are very supportive as well. When they know you’re doing your conversion they are a little bit more “Oh yeah come on. We’ll go through this, we’ll talk through that.” (p.400)

Example 5

Finding: A need to feel safe and at ease with sex in the new family situation: Reassurance [NS] (Baldwin et al. 2018).

Illustration of finding: ‘They experienced the visit to be quite focused on the baby and less on the relationship and on sexuality. The men perceived that it was difficult to get accurate information about the changes in the sexual relationship after childbirth.’ (p. 721)