Studies that are eligible for inclusion in a systematic review of measurement properties must be assessed for methodological quality. Two reviewers should conduct independent appraisals and then reach consensus on study ratings. A third reviewer may be consulted where necessary. We recommend using and citing the COSMIN Risk of Bias checklist (Mokkink et al., 2018a). The COSMIN Risk of Bias checklist can be downloaded from the COSMIN website: https://www.cosmin.nl/tools/guideline-conducting-systematic-review-outcome-measures.
The COSMIN Risk of Bias checklist is modular, containing 10 boxes with standards for PROM development and nine measurement properties (see Table below). The PROM development box only needs to be completed for studies that report on the development of a measurement instrument. Likewise, each measurement property box only needs to be completed if a particular study has assessed that property. It may be useful for reviewers to create a table following the template below to identify which boxes need completing for each study.
Instrument | Included Studies | Outcomes | |||||||||
PROM Development | Content Validity | Structural validity | Internal consistency | Cross-cultural validity/measurement invariance | Reliability | Measurement error | Criterion validity | Hypotheses testing for construct validity | Responsiveness | ||
Instrument A | Study 1 | x | x | x | x | x | |||||
Study 2 | x | x | x | ||||||||
Study 3 | x | x | x | ||||||||
Instrument B | Study 4 | x | x | x | x | ||||||
Study 5 | x | x | x | x | |||||||
Study 6 | x | x | x | x | x | ||||||
Instrument C | Study 7 | x | x | x | |||||||
Study 8 | x | x | x | ||||||||
Each criteria in the checklist is rated as either very good, adequate, doubtful, or inadequate. Studies then receive an overall rating for methodological quality, taking the lowest rating of any criteria (i.e. the worst score counts). A spreadsheet is also available for download via the COSMIN website which can be used to enter and organize the appraisal results (see ‘help organizing your COSMIN Risk of Bias ratings’ within the guideline for systematic reviews section: https://www.cosmin.nl/tools/guideline-conducting-systematic-review-outcome-measures/).