Expert opinion – whether expressed by an individual, by a learned body or by a group of experts in the form of a consensus guideline – draws on the experience of practitioners. It may also draw on the expertise of consumer representatives that are aligned with affiliated organizations. However, the opinion of experts is more than just their practical experience; it is based on their understanding of the knowledge and experience; moreover it is the expression of these opinions in writing, and publishing in journals, magazines, webpages, etc. So, we should also consider the risk of “speech bias” according to the circumstances in which they expressed their opinions.
Thus, validity in this context relates to the soundness of opinion in terms of its logic and its ability to convince, the authority of the source and the quality of the opinion that renders it supportable. Although expert opinion is non-research evidence, it is empirically derived and mediated through the cognitive processes of practitioners who have typically been trained in scientific methods.
The focus then of appraisal is on authenticity: specifically, authenticity of the opinion, its source, the possible motivating factors and how alternate opinions are addressed. It is also focused on the assessment of credibility of the expert voice, and decision as to whether the arguments are logical. The items of appraisal are standardized for this type of literature, while the methods are the same as for appraisal of any type of literature. Standardized appraisal criteria require the primary and secondary reviewer to meet or electronically discuss the criteria to ensure a common understanding, then to apply them individually to each paper. Once both primary and secondary reviewers have conducted appraisal, any discrepancies in opinion are discussed and a mutual decision agreed upon. It is JBI policy that all systematic reviews need to be critically appraised using the JBI SUMARI critical appraisal checklist for text and opinion papers.