Correspondence to:
Dr F Cluzeau, Department of Public Health Sciences, St George’s Hospital Medical School, Cranmer Terrace, London SW17 0RE, UK;
f.cluzeau{at}sghms.ac.uk

Background: International interest in clinical practice guidelines has never been greater but many published guidelines do not meet the basic quality requirements. There have been renewed calls for validated criteria to assess the quality of guidelines.

Objective: To develop and validate an international instrument for assessing the quality of the process and reporting of clinical practice guideline development.

Methods: The instrument was developed through a multi-staged process of item generation, selection and scaling, field testing, and refinement procedures. 100 guidelines selected from 11 participating countries were evaluated independently by 194 appraisers with the instrument. Following refinement the instrument was further field tested on three guidelines per country by a new set of 70 appraisers.

Results: The final version of the instrument contained 23 items grouped into six quality domains with a 4 point Likert scale to score each item (scope and purpose, stakeholder involvement, rigour of development, clarity and presentation, applicability, editorial independence). 95% of appraisers found the instrument useful for assessing guidelines. Reliability was acceptable for most domains (Cronbach’s alpha 0.64–0.88). Guidelines produced as part of an established guideline programme had significantly higher scores on editorial independence and, after the publication of a national policy, had significantly higher quality scores on rigour of development (p<0.005). Guidelines with technical documentation had higher scores on that domain (p<0.0001).

Conclusions: This is the first time an appraisal instrument for clinical practice guidelines has been developed and tested internationally. The instrument is sensitive to differences in important aspects of guidelines and can be used consistently and easily by a wide range of professionals from different backgrounds. The adoption of common standards should improve the consistency and quality of the reporting of guideline development worldwide and provide a framework to encourage international comparison of clinical practice guidelines.

Keywords: validation; quality; guidelines

This article has been cited by other articles:

				M. B. Henley, C. Turkelson, J. J. Jacobs, and R. H. Haralson AOA Symposium. Evidence-Based Medicine, the Quality Initiative, and P4P: Performance or Paperwork? J. Bone Joint Surg. Am., December 1, 2008; 90(12): 2781 - 2790. [Full Text] [PDF]

				G. Pentheroudakis, R. Stahel, H. Hansen, and N. Pavlidis Heterogeneity in cancer guidelines: should we eradicate or tolerate? Ann. Onc., December 1, 2008; 19(12): 2067 - 2078. [Abstract] [Full Text] [PDF]

				A. Qaseem, V. Snow, P. Shekelle, R. Hopkins Jr., D. K. Owens, and for the Clinical Efficacy Assessment Subcommittee Screening for HIV in Health Care Settings: A Guidance Statement From the American College of Physicians and HIV Medicine Association Ann Intern Med, November 30, 2008; (2008) 0000605-200901200-00300. [Abstract] [Full Text] [PDF]

				P. Alonso-Coello and on behalf of Mario Delgado-Noguera, Sera Tort, Ign Quality of guidelines on obesity in children is worrying BMJ, November 11, 2008; 337(nov11_2): a2474 - a2474. [Full Text]

				M. Carey, H. Buchan, and R. Sanson-Fisher The cycle of change: implementing best-evidence clinical practice Int. J. Qual. Health Care, November 6, 2008; (2008) mzn049v1. [Abstract] [Full Text] [PDF]

				E. Nagy, J. Watine, P. S. Bunting, R. Onody, W. P. Oosterhuis, D. Rogic, S. Sandberg, K. Boda, and A. R. Horvath Do Guidelines for the Diagnosis and Monitoring of Diabetes Mellitus Fulfill the Criteria of Evidence-Based Guideline Development? Clin. Chem., November 1, 2008; 54(11): 1872 - 1882. [Abstract] [Full Text] [PDF]

				E. P. Diamandis, B. R. Hoffman, and C. M. Sturgeon National Academy of Clinical Biochemistry Laboratory Medicine Practice Guidelines for the Use of Tumor Markers Clin. Chem., November 1, 2008; 54(11): 1935 - 1939. [Full Text] [PDF]

				W.L.D.M. Nelen, R.W. van der Pluijm, R.P.M.G. Hermens, C. Bergh, P. de Sutter, K.G. Nygren, A.M.M. Wetzels, R.P.T.M. Grol, and J.A.M. Kremer The methodological quality of clinical guidelines of the European Society of Human Reproduction and Embryology (ESHRE) Hum. Reprod., August 1, 2008; 23(8): 1786 - 1792. [Abstract] [Full Text] [PDF]

				M. Krahn and G. Naglie The Next Step in Guideline Development: Incorporating Patient Preferences JAMA, July 23, 2008; 300(4): 436 - 438. [Full Text] [PDF]

				F. Chiappelli and O. S. Cajulis Transitioning Toward Evidence-Based Research in the Health Sciences for the XXI Century Evid. Based Complement. Altern. Med., June 1, 2008; 5(2): 123 - 128. [Abstract] [Full Text] [PDF]

				P. Lindsay RN PhD, M. Bayley MD, A. McDonald BScPT, I. D. Graham PhD, G. Warner PhD, and S. Phillips MBBS Toward a more effective approach to stroke: Canadian Best Practice Recommendations for Stroke Care Can. Med. Assoc. J., May 20, 2008; 178(11): 1418 - 1425. [Abstract] [Full Text] [PDF]

				J. A. Kellum, R. L. Mehta, A. Levin, B. A. Molitoris, D. G. Warnock, S. V. Shah, M. Joannidis, C. Ronco, and for the Acute Kidney Injury Network (AKIN) Development of a Clinical Research Agenda for Acute Kidney Injury Using an International, Interdisciplinary, Three-Step Modified Delphi Process Clin. J. Am. Soc. Nephrol., May 1, 2008; 3(3): 887 - 894. [Abstract] [Full Text] [PDF]

				A. Davenport, C. Bouman, A. Kirpalani, P. Skippen, A. Tolwani, R. L. Mehta, and P. M. Palevsky Delivery of Renal Replacement Therapy in Acute Kidney Injury: What Are the Key Issues? Clin. J. Am. Soc. Nephrol., May 1, 2008; 3(3): 869 - 875. [Abstract] [Full Text] [PDF]

				A. Boivin, F. Legare, and M.-P. Gagnon Competing norms: Canadian rural family physicians' perceptions of clinical practice guidelines and shared decision-making J Health Serv Res Policy, April 1, 2008; 13(2): 79 - 84. [Abstract] [Full Text] [PDF]

				V. A. Palda, I. D. Graham, D. Davis, J. Burgers, M. Brouwers, F. Cluzeau, and On behalf of the AGREE research trust Guideline Adaptation: An Appealing Alternative to De Novo Guideline Development Ann Intern Med, April 1, 2008; 148(7): 564 - 564. [Full Text] [PDF]

				J. Dartnell, M. Hemming, J. Collier, and G. Ollenschlaeger Putting evidence into context: some advice for guideline writers Evid. Based Nurs., January 1, 2008; 11(1): 6 - 8. [Full Text] [PDF]

				D Acolet Quality of neonatal care and outcome Arch. Dis. Child. Fetal Neonatal Ed., January 1, 2008; 93(1): F69 - F73. [Abstract] [Full Text] [PDF]

				T. Nijsten Level of Agreement With the British Guidelines for the Use of Biological Therapies for Psoriasis Arch Dermatol, December 1, 2007; 143(12): 1567 - 1569. [Full Text] [PDF]

				J. Dartnell, M. Hemming, J. Collier, and G. Ollenschlaeger Putting evidence into context: some advice for guideline writers Evid. Based Med., October 1, 2007; 12(5): 130 - 132. [Full Text] [PDF]

				A. Qaseem, S. Vijan, V. Snow, J. T. Cross, K. B. Weiss, D. K. Owens, and for the Clinical Efficacy Assessment Subcommittee Glycemic Control and Type 2 Diabetes Mellitus: The Optimal Hemoglobin A1c Targets. A Guidance Statement from the American College of Physicians Ann Intern Med, September 18, 2007; 147(6): 417 - 422. [Abstract] [Full Text] [PDF]

				E. Ketola, M. Kaila, and M. Honkanen Guidelines in context of evidence Qual. Saf. Health Care, August 1, 2007; 16(4): 308 - 312. [Abstract] [Full Text] [PDF]

				J G Williams, S E Roberts, M F Ali, W Y Cheung, D R Cohen, G Demery, A Edwards, M Greer, M D Hellier, H A Hutchings, et al. Gastroenterology services in the UK. The burden of disease, and the organisation and delivery of services for gastrointestinal and liver disorders: a review of the evidence Gut, February 1, 2007; 56(suppl_1): 1 - 113. [Full Text] [PDF]

				J S Smolen, E C Keystone, P Emery, F C Breedveld, N Betteridge, G R Burmester, M Dougados, G Ferraccioli, U Jaeger, L Klareskog, et al. Consensus statement on the use of rituximab in patients with rheumatoid arthritis Ann Rheum Dis, February 1, 2007; 66(2): 143 - 150. [Abstract] [Full Text] [PDF]

				R. D. Kent Evidence-Based Practice in Communication Disorders: Progress not Perfection Lang Speech Hear Serv Sch, October 1, 2006; 37(4): 268 - 270. [Abstract] [Full Text] [PDF]

				K. Verkerk, H. Van Veenendaal, J. L. Severens, E. J. M. Hendriks, and J. S. Burgers Considered judgement in evidence-based guideline development Int. J. Qual. Health Care, October 1, 2006; 18(5): 365 - 369. [Abstract] [Full Text] [PDF]

				G. Elwyn, A. O'Connor, D. Stacey, R. Volk, A. Edwards, A. Coulter, R. Thomson, A. Barratt, M. Barry, S. Bernstein, et al. Developing a quality criteria framework for patient decision aids: online international Delphi consensus process BMJ, August 26, 2006; 333(7565): 417. [Abstract] [Full Text] [PDF]

				W Zhang and M Doherty EULAR recommendations for knee and hip osteoarthritis: a critique of the methodology Br. J. Sports Med., August 1, 2006; 40(8): 664 - 669. [Abstract] [Full Text] [PDF]

				E.C. Haagen, R.P.M.G. Hermens, W.L.D.M. Nelen, D.D.M. Braat, R.P.T.M. Grol, and J.A.M. Kremer Subfertility guidelines in Europe: the quantity and quality of intrauterine insemination guidelines Hum. Reprod., August 1, 2006; 21(8): 2103 - 2109. [Abstract] [Full Text] [PDF]

				B. Fervers, J. S. Burgers, M. C. Haugh, J. Latreille, N. Mlika-Cabanne, L. Paquet, M. Coulombe, M. Poirier, and B. Burnand Adaptation of clinical guidelines: literature review and proposition for a framework and procedure. Int. J. Qual. Health Care, June 1, 2006; 18(3): 167 - 176. [Abstract] [Full Text] [PDF]

				I. P. Bridevaux, A.-M. Silaghi, J.-P. Vader, F. Froehlich, J.-J. Gonvers, and B. Burnand Appropriateness of colorectal cancer screening: appraisal of evidence by experts Int. J. Qual. Health Care, June 1, 2006; 18(3): 177 - 182. [Abstract] [Full Text] [PDF]

				E. Clark, E. F. Donovan, and P. Schoettker From outdated to updated, keeping clinical guidelines valid Int. J. Qual. Health Care, June 1, 2006; 18(3): 165 - 166. [Full Text] [PDF]

				S. Leucht, M. Stiegler, C. Rummel, K. Wahlbeck, and W. Kissling Call for a European Guidelines Institute The British Journal of Psychiatry, February 1, 2006; 188(2): 193 - 193. [Full Text] [PDF]

				J. S. Burgers Guideline Quality and Guideline Content: Are They Related? Clin. Chem., January 1, 2006; 52(1): 3 - 4. [Full Text] [PDF]

				J. Watine, B. Friedberg, E. Nagy, R. Onody, W. Oosterhuis, P. S. Bunting, J.-C. Charet, and A. R. Horvath Conflict between Guideline Methodologic Quality and Recommendation Validity: A Potential Problem for Practitioners Clin. Chem., January 1, 2006; 52(1): 65 - 72. [Abstract] [Full Text] [PDF]

				E.C. Haagen, W.L.D.M. Nelen, R.P.M.G. Hermens, D.D.M. Braat, R.P.T.M. Grol, and J.A.M. Kremer Barriers to physician adherence to a subfertility guideline Hum. Reprod., December 1, 2005; 20(12): 3301 - 3306. [Abstract] [Full Text] [PDF]

				W. Gaebel, S. Weinmann, N. Sartorius, W. Rutz, and J. S. McIntyre Schizophrenia practice guidelines: international survey and comparison The British Journal of Psychiatry, September 1, 2005; 187(3): 248 - 255. [Abstract] [Full Text] [PDF]

				I. D Graham and M. B Harrison Evaluation and adaptation of clinical practice guidelines Evid. Based Nurs., July 1, 2005; 8(3): 68 - 72. [Full Text] [PDF]

				J. Vlayen, B. Aertgeerts, K. Hannes, W. Sermeus, and D. Ramaekers A systematic review of appraisal tools for clinical practice guidelines: multiple similarities and one common deficit Int. J. Qual. Health Care, June 1, 2005; 17(3): 235 - 242. [Abstract] [Full Text] [PDF]

				T. Ostbye, K. S. H. Yarnall, K. M. Krause, K. I. Pollak, M. Gradison, and J. L. Michener Is There Time for Management of Patients With Chronic Diseases in Primary Care? Ann. Fam. Med, May 1, 2005; 3(3): 209 - 214. [Abstract] [Full Text] [PDF]

				N. Boluyt, C. R. Lincke, and M. Offringa Quality of Evidence-Based Pediatric Guidelines Pediatrics, May 1, 2005; 115(5): 1378 - 1391. [Abstract] [Full Text] [PDF]

				B. Fervers, J. S. Burgers, M. C. Haugh, M. Brouwers, G. Browman, F. Cluzeau, and T. Philip Predictors of high quality clinical practice guidelines: examples in oncology Int. J. Qual. Health Care, April 1, 2005; 17(2): 123 - 132. [Abstract] [Full Text] [PDF]

				P. Littlejohns, G. Leng, and A. Sutcliffe NICE responds to criticism of hypertension guidelines BMJ, February 5, 2005; 330(7486): 309 - 309. [Full Text]

				G Ollenschlager, C Marshall, S Qureshi, K Rosenbrand, J Burgers, M Makela, J Slutsky, and for the Board of Trustees 2002, Guidelines Interna Improving the quality of health care: using international collaboration to inform guideline programmes by founding the Guidelines International Network (G-I-N) Qual. Saf. Health Care, December 1, 2004; 13(6): 455 - 460. [Abstract] [Full Text] [PDF]

				U. Lunder, S. Sauter, and C.-J. Furst Evidence-based palliative care: beliefs and evidence for changing practice Palliative Medicine, June 1, 2004; 18(4): 265 - 266. [PDF]

				J.S. Burgers, B. Fervers, M. Haugh, M. Brouwers, G. Browman, T. Philip, and F.A. Cluzeau International Assessment of the Quality of Clinical Practice Guidelines in Oncology Using the Appraisal of Guidelines and Research and Evaluation Instrument J. Clin. Oncol., May 15, 2004; 22(10): 2000 - 2007. [Abstract] [Full Text] [PDF]