5.1.3.3 Evidence from policy/consensus guidelines

5.1.2.3.1 Definition

Policy: A course or principle of action adopted or proposed by an organization or individual ¹⁶

Policy refers to policy documents or communication artefacts that generally, in healthcare, give direction for action. It relates to policies (guidelines, standard procedures or statements) at public, organizational or clinical levels, usually developed by an expert or group of experts or a government department on a healthcare matter. Sources of this type of data may come from the websites of government departments, consumer groups, professional associations or industry/provider groups.

Policy refers to a deliberate set of principles designed to guide decisions and achieve rational outcomes, in the form of consensus guidelines or policy statements. In healthcare, a policy or consensus guideline is essentially a statement of intent that is often then implemented as a procedure or protocol. 

The term is used in many different ways, varying from country to country, institution to institution, organization to organization and sometimes within institutions and organisations, but there are some central features common to all good policy:

• It states matters of principle;

• It is focused on action, stating what is to be done and by whom; and

• It is an authoritative statement, made by a person, group, organization or body with the power to do so.

Evidence-based policy making has been advocated across policy making systems at all levels since the emergence of the evidence-based healthcare movement, and policies at the operational level (i.e., within health units) is frequently evidence based. However, policy making at the national, state, regional and local levels is often strongly associated with political, professional and fiscal issues and a reliance on evidence is not always apparent. Policy and guideline documents at all levels generally involve key stakeholders in their development, including patients/clients, clinical experts and health service managers, and represent an investment of time, experience and expertise. Some policies and guidelines are rigorous in their reference to the evidence but many, although taking existing external evidence into account, focus on reaching, if not total consensus, at least a majority view of those involved in the policy development process. Whilst policy and guideline developers may commission rigorous systematic reviews and draw on them to formulate policy, many focus on policy or guidelines developed and published in other jurisdictions, or health units; thus, conducting a synthesis of consensus guidelines or of policy statements or documents is increasing.

Policies and guidelines are complex and may apply to entire populations in varied contexts and they need to consider issues related to implementation. Thus, the concept of evidence generally focuses on the best available data, and not the best possible data. Mays et al ²⁸, in a methodological article on systematic reviews aimed at informing decision makers and managers, argue that the more the authors of a knowledge review seek to support decision making, the more the review must consider context and the more open it must be to different forms of ‘evidence.’ ²⁸ This openness implies including quantitative and qualitative data, research data and other types of data.

The JBI method for synthesizing knowledge from policies and guidelines adopts openness toward data, going beyond the exploration of the scientific literature, to include exploration of the ‘grey’ literature (documents produced by governments or non-profit organizations, statements by professional associations, opinion polls, etc.).

Policies and Consensus Guidelines as Evidence

Although most policy documents and guidelines draw on formal external evidence, the synthesis of evidence embedded in them usually takes an essentially textual approach. That is, each policy piece retrieved for synthesis is regarded as textual data (much like expert opinion) that can be synthesized using a process of meta-aggregation. However, given the likelihood that a policy or guideline has referred to external evidence, the degree to which the text is supported by evidence is of some importance and can be accounted for in the critical appraisal stage of the synthesis.

The classical policy/guideline development process generally involves (but is not limited to):

• Identifying the need for the policy/guideline;

• Identifying existing local, national or international policies/guidelines and determining if they can be adopted without change or with some change;

• Establishing a policy/guideline development team or group, often including policy experts, subject experts, interested practitioners and appropriate consumers/service users;

• Conducting research and analysis and literature reviews to identify, evaluate and summarize the external evidence on the topic. The degree to which this occurs is variable, with some policies and guidelines relying entirely on the input of the development team or group and others focusing strongly on the external evidence. Some policies and guidelines consider external evidence but allow the expert opinion of development group members or political or financial imperatives to overrule the external evidence;

• Drafting the policy/guideline;

• Consultation with stakeholders, other experts and opinion leaders; and

• Finalizing the policy/guideline.

Thus, policies and guidelines are complex and variable in their content, rigour of development and intended influence (e.g. a whole country or a single health unit such as a hospital, ward or clinic). At one end of a continuum, they can be explicitly based on a thorough examination of the evidence whilst at the other, be focused entirely on the views and opinion of the policy’s/guideline’s developers and may in some cases be in conflict with the extant external evidence.

Kopp²⁹ posits that public policies and organizational policies pursue either a ‘top-down’ strategy or a ‘bottom-up’ strategy. Top-down policy occurs when policy-makers seek to introduce a new policy or modify existing policy, often because a problem requires a response. Although consultation and evidence gathering may or may not occur, policy-makers decide to change existing policy or introduce new policy because they want to address a problem they consider important in health or healthcare. Bottom-up policy is usually a response to campaigns or requests from clinicians, patients or others. These campaigns may be welcomed by policy-makers or resisted strongly, in which case the campaigners may have to invest a lot of time and energy. In addition, bottom-up campaigns may involve a variety of groups with different views or agendas, and the debate may become a competition between these groups, or the differences may lead to internal disputes.

Guidelines are usually systematically developed statements designed to inform, and sometimes direct, decision making in health service settings. Guidelines can also be used for public policy.  Policies and guidelines play an important role in healthcare delivery and the practices of healthcare professionals and, for our purposes as reviewers, are best categorized as:

• Public policies;

• Organizational policies; and

• Clinical/Practice guidelines.

Public policy

Public policy is a strategic action carried out by a public authority with an overall aim of promoting a particular phenomena. Examples of well-publicised public policy include policies on obesity, smoking, the role of the nurse practitioner in primary health care or the organization of maternity services. ³⁰

Organizational policy

Health services (national, regional and local) are responsible for providing policy and procedural guidelines that both reflect legislation and the ethical standards of the community and support the delivery of services and the practices of clinicians. Indeed, the quality-of-service delivery is dependent on the responsibility of both the organization and the worker in following the policies that guide service delivery. Organizational policies are influenced by the values and beliefs that the organization holds, and problems experienced by an organization, such as an increasing number of incidents where people with disabilities are discriminated against in the workplace.

Clinical/Practice Guidelines

A clinical practice guideline is a systematically developed statement to inform or direct clinical decision-making. Such guidelines are developed at a number of levels:

• At a national or State level, to inform or direct practices and services across the systems of the jurisdiction. 

• At an organizational level, such as a health district, a hospital or a local health system.

• At the local, service level such as a ward or a clinic.

Many clinical guidelines are explicitly based on the evidence, with some commissioning rigorous systematic reviews and others relying heavily on existing syntheses and systematic reviews. Clinical guidelines are defined as: “…statements that include recommendations intended to optimize patient care that are informed by a systematic review of the evidence and an assessment of the benefits and harms of alternative care options.” ^{31 (page 3)}

This is not as widely accepted as the National Academy of Medicine may think, with many guidelines developed by professional organizations and health services focusing much more on local data and the consensus of experts, and sometimes with no reference to the external evidence.³² Evidence-based clinical guidelines, though often robust in their development, do have limitations in that they are often based on low levels of evidence, they may be influenced by the guideline development team or group members and they may lack of information on new treatments.³³ The beliefs of guideline development team members, often clinical experts may, in spite of the evidence, draw on misconceptions and personal recollections that misrepresent reality and practices that are not in the best interests of the patient perspective may be recommended to help control costs, serve societal needs, or protect special interests.