Development, inter-rater reliability and feasibility of a checklist to assess implementation (Ch-IMP) in systematic reviews: the case of provider-based prevention and treatment programs targeting children and youth

BMC Medical Research Methodology
Margaret CargoKarin Hannes

Abstract

Several papers report deficiencies in the reporting of information about the implementation of interventions in clinical trials. Information about implementation is also required in systematic reviews of complex interventions to facilitate the translation and uptake of evidence of provider-based prevention and treatment programs. To capture whether and how implementation is assessed within systematic effectiveness reviews, we developed a checklist for implementation (Ch-IMP) and piloted it in a cohort of reviews on provider-based prevention and treatment interventions for children and young people. This paper reports on the inter-rater reliability, feasibility and reasons for discrepant ratings. Checklist domains were informed by a framework for program theory; items within domains were generated from a literature review. The checklist was pilot-tested on a cohort of 27 effectiveness reviews targeting children and youth. Two raters independently extracted information on 47 items. Inter-rater reliability was evaluated using percentage agreement and unweighted kappa coefficients. Reasons for discrepant ratings were content analysed. Kappa coefficients ranged from 0.37 to 1.00 and were not influenced by one-sided bias. Most kappa ...Continue Reading

References

Sep 1, 1967·Journal of Health and Social Behavior·E A Suchman
Feb 17, 2000·American Journal of Community Psychology·T W HarachiC B Fleming
Mar 10, 2000·Health Education & Behavior : the Official Publication of the Society for Public Health Education·T BaranowskiD T Wang
Apr 18, 2000·Health Education & Behavior : the Official Publication of the Society for Public Health Education·T Baranowski, G Stables
Feb 27, 2003·Journal of Consulting and Clinical Psychology·Sandra Jo WilsonJames H Derzon
Feb 15, 2005·Journal of Epidemiology and Community Health·J DoyleA Portela
Jun 4, 2005·Injury Prevention : Journal of the International Society for Child and Adolescent Injury Prevention·L AraiJ Popay
Dec 21, 2005·Prevention Science : the Official Journal of the Society for Prevention Research·Brian R FlayPeter Ji
Jan 27, 2007·The Cochrane Database of Systematic Reviews·E A KristjanssonP Tugwell
Mar 23, 2007·American Journal of Health Promotion : AJHP·Sheila Giardini MurtaBrian Oldenburg
Oct 24, 2007·BMJ : British Medical Journal·Trisha GreenhalghVivian Robinson
Dec 7, 2007·Implementation Science : IS·Christopher CarrollShashi Balain
Jan 22, 2008·Journal of Public Health·Rebecca ArmstrongPenelope Hawe
Jun 28, 2008·BMJ : British Medical Journal·Paul GlasziouSasha Shepperd
Jul 23, 2008·The Cochrane Database of Systematic Reviews·Maree L HackettChristina Halteh
Oct 1, 2008·BMJ : British Medical Journal·Peter CraigUNKNOWN Medical Research Council Guidance
Nov 27, 2008·Psychotherapy Research : Journal of the Society for Psychotherapy Research·Ladislav Timulak
Mar 26, 2010·BMC Medicine·Kenneth F SchulzUNKNOWN CONSORT Group
Jan 5, 2011·Administration and Policy in Mental Health·Gregory A AaronsSarah McCue Horwitz
Sep 17, 2011·Journal of Children's Services·Wendi F Cross, Jennifer C West
Aug 21, 2013·American Journal of Public Health·Paul MontgomerySean Grant
Sep 6, 2013·Implementation Science : IS·Paul MontgomeryEvan Mayo-Wilson
Sep 12, 2013·Journal of Epidemiology and Community Health·Graham MooreJanis Baird
Feb 25, 2014·The Journal of Adolescent Health : Official Publication of the Society for Adolescent Medicine·Amy Lynn Margolis, Allison Yvonne Roper
Feb 25, 2014·The Journal of Adolescent Health : Official Publication of the Society for Adolescent Medicine·Allison Metz, Bianca Albers
Mar 1, 2011·Research Synthesis Methods·Laurie M AndersonPeter Tugwell

❮ Previous
Next ❯

Citations

Dec 9, 2017·Drug Safety : an International Journal of Medical Toxicology and Drug Experience·Meredith Y SmithElaine H Morrato
Dec 21, 2017·International Journal of Environmental Health Research·Emmy De BuckTaryn Young
Dec 4, 2019·International Journal of Nursing Practice·Zann SohVivien Xi Wu
Jun 5, 2018·BioMed Research International·Janet HarrisMichael Wright

❮ Previous
Next ❯

Software Mentioned

Ch
- Reviewer
IMP
NVivo
Reviewer
EPPI

Related Concepts

Related Feeds

Carbapenems (ASM)

Carbapenems are members of the beta lactam class of antibiotics and are used for the treatment of severe or high-risk bacterial infections. Discover the latest research on carbapenems here.

Carbapenems

Carbapenems are members of the beta lactam class of antibiotics and are used for the treatment of severe or high-risk bacterial infections. Discover the latest research on carbapenems here.