Commentary
Guideline panels should seldom make good practice statements: guidance from the GRADE Working Group

https://doi.org/10.1016/j.jclinepi.2016.07.006Get rights and content

Introduction

Good practice statements represent recommendations that guideline panels feel are important but that, in the judgment of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group, are not appropriate for formal ratings of quality of evidence (synonyms: certainty or confidence in evidence). Building on the preliminary guidance published in the Journal of Clinical Epidemiology series [1], members of the GRADE working group have recently published a discussion of the methodologic challenges associated with good practice statements [2]. When presented to the GRADE working group, the suggested approach generated controversy. Although there were other less compelling concerns, the primary concern was that guideline panels are at risk of overusing good practice statements and the article did not provide sufficient safeguards to protect against such overuse. This article to a considerable extent duplicates the previous published guidance. There are, however, important differences, and the current version represents official GRADE guidance and will be included in the GRADE handbook [3].

Guideline panels may present what we would interpret as good practice statements as strong recommendations based on low-quality evidence, or do so formally and explicitly [4], [5], [6], [7]. We hope that the guidance herein results in more limited and appropriate use of good practice statements and, when panels feel compelled to make such statements, improves their use.

Panels using GRADE to address recommendation such as those in Box 1 typically offer strong recommendations and classify the evidence as low or very low quality. Such recommendations are common: a systematic examination of Endocrine Society recommendations revealed that of 121 strong recommendations based on low- or very low–quality evidence (discordant recommendations), 43 (36%) were best categorized as good practice statements [12]. In a similar examination of World Health Organization recommendations, of 160 discordant recommendations, 29 (18%) were classified as good practice statements [13]. We would argue that the evidence underlying these recommendations is actually high quality—the key to our suggested reclassification.

Box 1 presents four good practice statements reported as GRADEd recommendations in different clinical practice guidelines. One strategy for recognizing a recommendation best characterized as a good practice statement is to ask: is the unstated alternative absurd or clearly not conforming to ethical norms (e.g., it would be absurd not to make patients with chronic noncancer pain aware of nonopioid treatments—see Box 1)? If the answer is yes, you are facing a possible good practice statement.

A somewhat more sophisticated strategy is to consider whether high-certainty indirect evidence that would be onerous and time consuming to formally accumulate and review supports the recommendation. Again, if the answer is yes, you are looking at a good practice statement.

Good practice statements characteristically represent situations in which a large and compelling body of indirect evidence, made up of linked evidence including several indirect comparisons, strongly supports the net benefit of the recommended action. By linked evidence, we mean that several separate bodies of evidence together allow inferences regarding net benefit (e.g., evidence regarding the accuracy of a diagnostic test and evidence regarding the effectiveness of treatment instituted on the basis of the test) [14], [15], [16], [17].

We have noted that guideline panels often apply the GRADE process to what are potentially good practice statements by grading the recommendation as strong, apparently reflecting high certainty in an intervention's effect. At the same time, they rate the quality of evidence as low or very low. This contradiction highlights a common misunderstanding of GRADE methodology. In the absence of randomized trials—indeed, in the absence of any formal studies addressing the question of interest—guideline panels often believe that they should classify evidence as low or very low quality. Intuitively, however, they recognize—in accord with GRADE—that indirect evidence can support a rating of high-quality evidence [18].

Panels should consider making good practice statements when they have high confidence that indirect evidence undoubtedly supports net benefit and when, in addition, it would be an onerous and unproductive exercise and thus a poor use of the panel's limited resources to collect this evidence.

For instance, why are we confident that it is wise to monitor patients with congenital adrenal hyperplasia for glucocorticoid excess (Box 1)? The reason is that relevant symptoms and signs appear not infrequently, that patients will suffer if clinicians fail to recognize these signs, and that clinical action can ameliorate the problem.

Consider again the advice regarding monitoring in patients with congenital adrenal hyperplasia. It would be possible to accumulate and summarize the relevant evidence. There have been no randomized trials or observational studies that have directly compared monitoring to no monitoring of glucocorticoid excess in patients with congenital adrenal hyperplasia—thus, we have no direct research evidence.

The panel could, nevertheless, build a case for the benefits through indirect evidence. They could collect all the reports of the adverse consequences of glucocorticoid excess. They could then collect the evidence that supports the usefulness of the relevant symptoms, signs, and laboratory tests in the diagnosis of glucocorticoid excess. Then, they could collect and summarize the evidence of the benefits of the candidate management strategies. Finally, they could link these three bodies of evidence to make the case for their high level of certainty regarding the net benefits of monitoring for glucocorticoid excess. The case for a good practice statement rather than a GRADEd recommendation is the poor use of time in collecting and summarizing the relevant evidence.

Why the confidence in providing, to sex workers, access to health services with evidence of more good than harm? First, confidence is based on an underlying ethical value we place in equitable access to health care. Second, because a large number of health care interventions do more good than harm, sex workers will therefore have better health if they have access to services.

In each case, although there is a great deal of evidence suggesting that the net benefits of the intervention are large, teasing out the nature of this evidence would be onerous. Given that time and energy is typically at a high premium in the guideline development exercise, their expenditure in turning good practice statements into GRADEd recommendations (strong recommendations based on high-, not low-, or very low–quality evidence) is likely to be inadvisable.

Good practice statements potentially allow guideline panels to issue strong recommendations that may be unwarranted [12], [19]. Furthermore, judgments about what are incontestable net benefits are inevitably subjective. Thus, guideline panels should use them sparingly.

GRADE suggests that guideline panels explicitly address the following issues before they make good practice statements (Table 1). We suggest that each good practice statement be accompanied by formal documentation that includes responses to each question in Table 1. Working through this formal documentation, panels may find reasons not to make the statement under consideration: they are unable to construct a clear and actionable statement; the statement is unnecessary; the net benefit is not as clear as they had initially thought; or collecting the necessary information is feasible. If they do proceed, users of the guideline will have a available clear rationale connecting the indirect evidence available to support the good practice statement.

First, among the necessary conditions for a good practice statement is that, as with all recommendations, good practice statements should be clear, specific—including specification of the population of interest—and actionable. For instance, in the statement in the Box 1 regarding congenital adrenal hyperplasia, the associated text should specify the frequency and nature of the monitoring, and the action to be taken should the clinician identify signs of glucocorticoid excess.

Note that, however, if monitoring would require multiple additional physician visits or the use of an expensive test, the net benefit of monitoring would be called into question, and this would no longer be a good practice statement. This highlights the necessity for, in good practice statements, clearly specifying the intervention and alternative, as well as giving careful thought to the downstream consequences of the action proposed. Such clear specification and careful thought may lead panels to conclude that a GRADEd recommendation, rather than a good practice statement, is warranted.

Second, the message should be necessary: that is, without the guidance, clinicians might fail to take the appropriate action. Knowledge that practice among the clinicians who represent the target audience is suboptimal would be the best way to satisfy this criterion. On the other hand, it may not be plausible that clinicians who, for instance, look after patients with congenital adrenal hyperplasia, will fail to monitor for signs of glucocorticoid excess. If it is not plausible, there is no need for the good practice statement.

Third, the net benefit should be large and unequivocal. Good practice statements will be most suitable when benefits are large and harm very small; certainty of benefits and harms are great; the values and preferences are clear; the intervention is cost saving; and the intervention is clearly acceptable, feasible, and promotes equity [20], [21], [22].

The fourth criterion that it is a poor use of a guideline panel's time and resources to collect and link the indirect evidence is an issue of opportunity cost: is the guideline panel's limited time and energy better spent on other efforts to maximize the guideline's methodologic quality and overall trustworthiness? With regard to this criterion, consider the following recommendation in which the criterion is not fulfilled: Women with severe hypertension during pregnancy should receive treatment with antihypertensive drugs. A guideline panel issued this as a strong recommendation based on very low–quality evidence [23]. If the panel really did believe the quality of evidence was very low (i.e., they were very uncertain there was net benefit), they should not have made a strong recommendation.

Is it possible, however, that the panel actually was sure there were important benefits (i.e., they really believed the evidence warranted high certainty) and was misapplying GRADE in the certainty judgment? If so, should this recommendation be transformed into a good practice statement?

The answer is that it should not. Presumably, the panel's logic starts with the fact that we have evidence warranting high certainty that, in nonpregnant individuals, treating severe hypertension over long periods results in important benefits in morbidity and mortality. This evidence is easy to find and summarize. The panel is then presumably deducing that treatment of pregnant individuals over shorter periods may also reduce long-term morbidity and possible mortality. The certainty that is warranted by this deduction might be a matter of debate but should be made explicit. If only low or very low certainty is warranted, a weak recommendation is appropriate. In any case, the recommendation requires a formal application of the GRADE approach.

Finally, given the subjective nature of the judgment that appreciable net benefit from the recommended behavior is incontestable, the rationale for that judgment should be explicit. The rationale should include an explicit statement of the chain of evidence that supports the recommendation. Earlier in this article, we have provided such rationales for two of the best practice statements in the Box 1. The third might be “The relation between physiological measures and patients' function in heart failure is weak. Patients value their function highly, and management should be tailored to optimizing function. Without an inquiry into function, such tailored management will not be possible.” The explicit statement of the rationale for the belief in benefit allows that judgment to be open to question.

For the fourth statement in the Box 1, the rationale might be as follows. “Opioids have serious potential adverse effects, including death, which are not associated with other interventions for addressing chronic noncancer pain. Thus, clinicians and patients should explore all effective and less toxic interventions before embarking on a trial of opioid therapy. Potential treatment includes behavioral interventions of which many patients may be unaware. Thus, the exploration of the full range of alternatives is very likely to ultimately decrease adverse effects associated with opioid therapy.”

Guideline users must be able to clearly distinguish whether a formal GRADE process underlies a particular recommendation. To clearly distinguish good practice statements, they should have a separate heading and be worded differently than GRADEd recommendations that are most commonly presented as “we recommend” for strong and weak recommendations and “we suggest” for weak (sometimes called conditional) recommendations. Adding the word “Ungraded” can also make this issue more explicit. Thus, the statements would be described as “ungraded good practice statement”. Box 2 presents examples of appropriate alternative wording. Whether differentiating good practice statements from graded recommendations is important enough that they should be placed in another part of the guideline is a matter of debate (or for electronic documents, have a separate link). In any case, as for GRADEd recommendations, the rationale for each good practice statement should only be a mouse-click away [24].

Section snippets

Conclusion

Members of the GRADE working group are uniformly concerned about the inappropriate use of good practice statements. Some members are so concerned they feel GRADE is unwise to provide guidance for such statements. Two considerations have motivated us to, nevertheless, proceed: one is that panels will be making such statements with or without GRADE guidance; and second that there may be unusual but legitimate reasons for making such statements.

For instance, a panel addressing chronic noncancer

Acknowledgments

The authors would like to express our appreciation to Laura Fochtmann for her extremely useful editing suggestions that improved the presentation of the material.

First page preview

First page preview
Click to open first page preview

References (24)

  • A. Kristiansen et al.

    Development of a novel, multilayered presentation format for clinical practice guidelines

    Chest

    (2015)
  • H. Schünemann et al.

    Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach

    (2015)
  • Cited by (149)

    View all citing articles on Scopus
    View full text