No short-cut in assessing trial quality: a case study.

Trials
Karim F Hirji

Abstract

Assessing the quality of included trials is a central part of a systematic review. Many check-list type of instruments for doing this exist. Using a trial of antibiotic treatment for acute otitis media, Burke et al., BMJ, 1991, as the case study, this paper illustrates some limitations of the check-list approach to trial quality assessment. The general verdict from the check list type evaluations in nine relevant systematic reviews was that Burke et al. (1991) is a good quality trial. All relevant meta-analyses extensively used its data to formulate therapeutic evidence. My comprehensive evaluation, on the other hand, brought to the surface a series of serious problems in the design, conduct, analysis and report of this trial that were missed by the earlier evaluations. A check-list or instrument based approach, if used as a short-cut, may at times rate deeply flawed trials as good quality trials. Check lists are crucial but they need to be augmented with an in-depth review, and where possible, a scrutiny of the protocol, trial records, and original data. The extent and severity of the problems I uncovered for this particular trial warrant an independent audit before it is included in a systematic review.

References

Sep 10, 1977·British Medical Journal·D Greenwood, F O'Grady
Jun 1, 1992·Clinical Otolaryngology and Allied Sciences·J Q ClaessenG J Hordijk
Sep 7, 1991·BMJ : British Medical Journal·P BurkeJ Dunleavey
Sep 27, 1967·Annals of the New York Academy of Sciences·C HalstedE Wolinsky
Dec 12, 1981·Lancet
Dec 1, 1983·The New England Journal of Medicine·T C ChalmersH Smith
Jun 10, 1982·The New England Journal of Medicine·R D Feigin
Aug 1, 1994·Journal of Clinical Epidemiology·J C Sinclair, M B Bracken
Jun 8, 1995·The New England Journal of Medicine·S Berman
Feb 1, 1995·JAMA : the Journal of the American Medical Association·K F SchulzD G Altman
Jul 13, 1994·JAMA : the Journal of the American Medical Association·M K Cho, L A Bero
Mar 25, 1996·Archives of Internal Medicine·K S KhanA Jadad
Jan 1, 1996·International Journal of Technology Assessment in Health Care·D MoherP Tugwell
Jan 1, 1997·Diagnostic Microbiology and Infectious Disease·R Cohen
Dec 5, 1997·JAMA : the Journal of the American Medical Association·L Culpepper, J Froom
Oct 31, 1998·BMJ : British Medical Journal·B Thornley, C Adams
Sep 3, 1999·American Journal of Epidemiology·C Poole, S Greenland
Sep 24, 1999·JAMA : the Journal of the American Medical Association·P JüniM Egger
Sep 24, 1999·JAMA : the Journal of the American Medical Association·J A Berlin, D Rennie
Jul 7, 2001·BMJ : British Medical Journal·P JüniM Egger
Aug 8, 2001·Current Oncology Reports·C L Bennett, J R Adams
Jun 1, 2002·JAMA : the Journal of the American Medical Association·Douglas G Altman
Jun 8, 2002·JAMA : the Journal of the American Medical Association·Ethan M BalkJoseph Lau
Nov 30, 2002·Lancet·Marissa Lassere, Kent Johnson
May 2, 2003·Human Reproduction·A Vail, E Gardener
Feb 20, 2004·The Cochrane Database of Systematic Reviews·P P GlasziouM Hayem
May 27, 2004·JAMA : the Journal of the American Medical Association·An-Wen ChanDouglas G Altman
Jul 16, 2004·Evidence-based Nursing·Stephanie Wright
Sep 29, 2004·CMAJ : Canadian Medical Association Journal = Journal De L'Association Medicale Canadienne·An-Wen ChanDouglas G Altman
Mar 30, 2005·Lancet·An-Wen Chan, Douglas G Altman
Jul 28, 2006·PLoS Clinical Trials·Peter M Rothwell
Aug 16, 2006·Journal of Evaluation in Clinical Practice·Navindra Persaud, Muhammad M Mamdani
Jan 20, 2007·BMJ : British Medical Journal·Harlan M KrumholzDavid S Egilman
Mar 29, 2007·PLoS Medicine·David MoherDouglas G Altman
May 23, 2007·International Journal of Epidemiology·J PildalP C Gøtzsche

❮ Previous
Next ❯

Citations

May 15, 2010·Herz·Christian A SchneiderErland Erdmann
Apr 21, 2011·Current Osteoporosis Reports·E Michael Lewiecki
Aug 12, 2010·BMJ : British Medical Journal·Peter C GøtzscheAndreas Lundh
Jan 9, 2009·Trials·Peter C Gøtzsche
Aug 10, 2011·Trials·Karim F Hirji, Zulfiqarali G Premji
May 29, 2014·Plastic and Reconstructive Surgery·Maria A RodriguesLydia M Ferreira
Jul 16, 2015·International Journal of Epidemiology·Graham D ColeDarrel P Francis
Jul 4, 2017·International Journal of Obesity : Journal of the International Association for the Study of Obesity·S ParamasivanJ L Donovan
Mar 26, 2011·Metabolic Brain Disease·Clarissa A RozaEmilio L Streck
Sep 25, 2016·Human Reproduction·Jack WilkinsonAndy Vail
Mar 6, 2021·International Journal of Gynecological Cancer : Official Journal of the International Gynecological Cancer Society·Kristen MoloneyAndreas Obermair

❮ Previous
Next ❯

Methods Mentioned

BETA
dissection

Related Concepts

Related Feeds

Antifungals

An antifungal, also known as an antimycotic medication, is a pharmaceutical fungicide or fungistatic used to treat and prevent mycosis such as athlete's foot, ringworm, candidiasis, cryptococcal meningitis, and others. Discover the latest research on antifungals here.

Antifungals (ASM)

An antifungal, also known as an antimycotic medication, is a pharmaceutical fungicide or fungistatic used to treat and prevent mycosis such as athlete's foot, ringworm, candidiasis, cryptococcal meningitis, and others. Discover the latest research on antifungals here.