1

From levels of evidence to grades of recommendation – The GRADE Working Group

Yngve Falck-YtterDeutsches Cochrane Zentrum

EBM in der Inneren Medizin13. Januar 2004

2

3

4

5

Hintergrund

Wieweit können wir Aussagen trauen?• Experten vs. Evidenz

• Narrativ vs. Systematische Reviews

• Explizit vs. Implizit

Sollen wir Aussagen als Empfehlungen übernehmen? Spielt dabei ein „level of evidence“ eine Rolle?

6

Hintergrund

Klinische Entscheidungen sind komplex:• Soll ich eine Kalziumarme Diät bei Nierensteinen

verordnen, nur weil es einleuchtet?

• Soll ich bei vaso-vagaler Synkope einen Herzschrittmacher implantieren? Welche outcomes sind wichtig? Habe mehr positive Effekte als negative (harm)? Wie stehen limitierte Ressourcen im Verhältnis zum Benefit?

Daher brauchen wir klinische Leitlinien

7

Hintergrund

Evidenzhierarchien• Canadian Task Force on Periodic Health Examinations

• United States Preventive Services Task Force

• Oxford Center of Evidence Based Medicine

• Scottish Intercollegiate Guidelines Network

• American Heart Association Werden jedoch uneinheitlich erstellt Verwenden uneinheitliche Empfehlungskategorien

(z.B.: II-2 = B = C+ = strong evidence = strongly recommended)

Oxford Centre of Evidence Based Medicine; http://www.cebm.net

8

Example

Level of evidence

I

II

III

IV

V

Studies on therapy

RCTs

Cohort studies

Case-control-studies

Case-series

Expert opinion

A

Recom-mendation

B

C

D

9

From Levels to Grades

Decreased grades 1 or 2 steps - 7%

Increased grades from

C to A 27%

Increased grades from

B to A or C to B 21%

Increasedgrades 1 or 2 steps48%

Grades equal

to levels45%

Up- and downgrading of grades of recommendation in clinical practice guidelines from the Association of the Scientific Medical Societies in Germany (n = 10/>1200 Clinical Practice Guidelines; total number of recommendations: 295 [mean: 30; range 5 to 61]. Levels of evidence: I = randomized controlled trials, II = observational studies, III = expert opinion; grades of recommendation: A = good evidence, B = fair evidence, C = poor evidence).

Committee of Ministers of the Council of Europe. Oct 2001.

10

11

Ziele von GRADE

Simple Sensible Explicit Reliable Address shortcoming of other schemes

(e.g explicitly include trade offs between benefit and risks)

12

Definitionen

Qualität der Evidenz: Ausmaß des Vertrauens in die Richtigkeit des Effektschätzers (extent to which one can be confident that an estimate of effect is correct)

Härtegrad einer Empfehlung:Ausmaß des Vertrauens, dass das Befolgen der Empfehlung mehr Nutzen wie Schaden ermöglicht (extent to which one can be confident that adherence to the recommendation will do more good than harm)

13

Grundlegender Ansatz

5 Schritte:

1. Qualität der Evidenz jedes wichtigen Outcomes der Studien Quality of evidence across studies for each important outcome

2. Welche Outcomes sind kritisch für eine Entscheidung Which outcomes are critical to a decision

3. Ermittelte Qualität der Evidenz dieser kritischen OutcomesThe overall quality of evidence across these critical outcomes

4. Abwägung von Nutzen und SchadenThe balance between benefits and harm

5. Härtegrad der EmpfehlungStrength of recommendation

14

1. Qualität der Evidenz jedes wichtigen Outcomes

a) Studiendesign

• RCTs vs Beobachtungsstudien

b) Studienqualität

• Critical appraisal (alloc. concealment, blinding, f/u…)

c) Übereinstimmung

• Gleichgerichteter Effekt aller Studien (Homogenität)

d) Vergleichbarkeit

• Personen, Interventionen & Outcome sind vergleichbar

e) Und…

• Spärliche Daten, hohes Risiko eines Publikationsbias

15

Quality of evidence across outcomes

Observational studies Quality of evidence

Randomized trials

HighExtremely strong

association and no major threats to validity

No serious flaws in study quality

Strong, consistent association and no

plausible confounders

Serious flaws in design or execution or quasi-

randomized trials

Moderate

LowNo serious flaws in

study qualityVery serious flaws in design or execution

Very lowSerious flaws in design

and executionVery serious flaws and

at least one other serious threat to validity

16

...continued Zusätzliche Faktoren, die die Qualität verringern:

• Important inconsistency of results• Uncertainty about directness

• Some uncertainty• Major uncertainty can lower the quality by two levels

• High probability of reporting bias• Sparse data

Zusätzliche Faktoren, die die Qualität vergrößern:• All plausible residual confounding, if present, would

already have reduced the observed effect• Evidence of a dose-response gradient

17

2. Outcomes critical to a decision

Outcomes may be important, but sometimes not critical to a decision

Only outcomes critical to a decision should provide a basis for recommendation

If information on harm is critical, it should be included even if uncertainty exists

18

3. Overall quality of evidence

The lowest quality of evidence for any critical outcome should provide the basis for grading

However, if evidence favors the same alternative and there is high quality for some but not all of those outcomes, overall quality should still be high

Weak evidence about implausible putative harms should not lower the overall grade of evidence

19

4. The balance between benefits and harm

Net benefits: Clearly does more good than harm Trade-offs: Important trade-offs between benefits

and harm Uncertain trade-offs: It is not clear whether the

intervention does more good than harm No net benefits: Clearly does not do more good than

harm

20

5. Grades of recommendation

„Do it“ or „Don‘t do it“• Indicating a judgment that a majority of well informed

people will make the same choice• Medical practice is expected to not to vary much

„Probably do it“ or „Probably don‘t do it“• Indicating a judgment that a majority of well informed

people will make the same choice, but a substantial minority will not

• Medical practice is expected to vary to some degree