9.3.7.4 Data collection

Standardized data extraction tools maximize the consistent extraction of accurate data across the included studies and are required for JBI Umbrella Reviews. The review should detail what data the reviewers extracted from the included systematic reviews and the JBI data extraction tool for Systematic Reviews and Research Syntheses must be appended to the review report (see Appendix 10.3). As mentioned , individual study level data should not be reported in an Umbrella Review (except where an outcome is only informed by one included study); the focus of reporting should be the results and findings of the included syntheses. Using the JBI extraction tool, at a minimum, details and data relevant to the items listed below should be extracted where the information is available. The majority of this information will appear in the Table of Included Study Characteristics to be appended to the review report, while other important details extracted, particularly relevant to the findings of the review (see below) will appear in the body of the review report:

Author/year

The citation details of included studies should be consistently referred to  throughout the document. The citation details should include the name of the first author (Vancouver reference) and year of publication.

Objective(s)

A clear description of the objective of the included research synthesis should be stated.

Participants (characteristics/total number)

The defining characteristics of the participants in studies included in the research syntheses should be detailed, for example this may include diagnostic criteria, or age or ethnicity. The total number of participants that inform the outcomes relevant to the Umbrella Review question from all studies included studies should be presented also.

Setting/context

Details of the setting of interest such as acute care, primary health care, or the community or a particular geographical location should be included. For some Umbrella Reviews, particularly those that draw upon qualitative research syntheses, the context that underpins the review question will be important to clearly reveal to the reader and may include but is not limited to consideration of cultural factors such as geographic location and specific racial or gender based interests.

Interventions/phenomena of interest

Clear, succinct details of the interventions or phenomena of interest should be described, including the type of intervention, the frequency and/or intensity of the intervention for example. A statement of the phenomena of interest is also required where applicable.

Number of databases/sources searched

The number of sources searched should be reported. Though this will have been considered during critical appraisal of the research synthesis, reporting to the reader of the review will allow rapid and easy comparison between differences of included reviews and also consideration of potential for publication bias in the event no formal analysis has been conducted. Where possible the names of databases and sources should be listed (i.e. if <5-10). The search range of each database should also be included.

Date range of included studies

The date range spanning from the earliest study that informs the included research synthesis to the latest should be reported. This is important information that allows for consideration of the currency of the evidence base not necessarily reflected in the year of publication of the research synthesis. If this is not readily identifiable in the table of study characteristics provided by the included synthesis, it should be discer nable by scanning the date range of publications through the results section of the included review.

Number of Studies/Type of Studies/Country of origin of included studies

Summary descriptive details of the included studies in the research synthesis should be reported. This includes the number of studies in the included research synthesis, the types of study designs included in the research synthesis, for example randomized controlled trials, prospective cohort study, phenomenology, ethnography etc., and also the country of origin of the included studies. The latter is important to allow the reader of the review to consider the external validity and generalizability of the results presented.

Appraisal instrument and rating

The instrument or tool used to assess risk of bias, rigor or study quality should be reported along with some summary estimate of the quality of primary studies in the included research synthesis. For example, for Umbrella Reviews that use the Jadad Scale, a mean score for quality may be reported whereas for checklist appraisals, reporting of cut-off score or any ranking of quality should be reported. An example of the latter would be exclusion of studies that score <3/10, and inclusion of four moderate quality studies (4-6/10) and two high quality studies (7-10/10).

Type of Review/Method of analysis

The type of research synthesis as stated by the authors of the included review should be detailed. The method of analysis or synthesis used by the included research synthesis should be reported. For example, this may include random effects meta-analysis, fixed effect meta-analysis, meta- aggregative synthesis or meta-ethnography.

Outcome(s)

Included here should be the outcomes of interest to the Umbrella Review question reported on by the research synthesis, i.e. the names or labels of the outcomes (see below for presentation of results).

Results/findings

The relevant findings or results presented by the included research syntheses must be extracted. For quantitative reviews, this will ideally be an effect estimate or measure from a presented meta- analysis. Measures of heterogeneity should also be extracted where applicable. In the absence of this a statement indicating the key result relevant to an outcome may be inserted in the required field. For qualitative syntheses, the key synthesized finding should be extracted.

Comments

There should be provision to extract and present in the table of included study characteristics any relevant details or comments on the included research synthesis by the authors of the Umbrella Review. These comments may be relevant details regarding the included research synthesis, for example, the congruence between the review results and conclusions, and for highlighting any potential methodological differences between the individual included reviews.