A confidence interval approach to sample size estimation for interobserver agreement studies with multiple raters and outcomes

Journal of Clinical Epidemiology
Michael A Rotondi, Allan Donner

Abstract

Studies measuring interobserver agreement (reliability) are common in clinical practice, yet discussion of appropriate sample size estimation techniques is minimal as compared with clinical trials. The authors propose a sample size estimation technique to achieve a prespecified lower and upper limit for a confidence interval for the κ coefficient in studies of interobserver agreement. The proposed technique can be used to design a study measuring interobserver agreement with any number of outcomes and any number of raters. Potential application areas include: pathology, psychiatry, dentistry, and physical therapy. This technique is illustrated using two examples. The first considers a pilot study in oral radiology, whose authors studied the reliability of the mandibular cortical index as measured by three dental professionals. The second example examines the level of interobserver agreement among four nurses with respect to five triage levels used in the Canadian Triage and Acuity Scale. This method should be useful in the planning stages of an interobserver agreement study in which the investigator would like to obtain a prespecified level of precision in the estimation of κ. An R software package (R Foundation for Statistical...Continue Reading

References

Jun 1, 1970·The British Journal of Psychiatry : the Journal of Mental Science·A E Maxwell
Feb 1, 1994·Scandinavian Journal of Dental Research·E KlemettiH Kröger
Feb 1, 1996·Stroke; a Journal of Cerebral Circulation·K BergerJ E Buring
Aug 28, 2002·The Journals of Gerontology. Series A, Biological Sciences and Medical Sciences·D Knezović ZlatarićP Kobler
Mar 3, 2004·Nutrition·J Mary Jones

❮ Previous
Next ❯

Citations

Mar 13, 2014·International Journal of Nursing Studies·Nicola ParentiDawn Dowding
Dec 4, 2014·Journal of Psychosomatic Research·Ka-Fai ChungChi-Wa Kwok
Mar 8, 2016·European Spine Journal : Official Publication of the European Spine Society, the European Spinal Deformity Society, and the European Section of the Cervical Spine Research Society·Julio UrrutiaCarlos Prada
Nov 15, 2014·The Journal of Manual & Manipulative Therapy·Afshin Heidar AbadyMichael A Rotondi
Apr 22, 2014·Journal of Psychosomatic Research·Ka-Fai ChungChi-Wa Kwok
Oct 23, 2016·Journal de gynécologie, obstétrique et biologie de la reproduction·H LemoineC Vayssière
Sep 14, 2018·Acta ortopedica brasileira·Sergio Charles LozoyaAndrea Alcázar Juárez
Apr 13, 2019·JAMA Network Open·Gustavo Bigaton LovadiniEdison Iglesias de Oliveira Vidal
Aug 2, 2019·The Journal of Nutrition, Health & Aging·H KellerH Chaudhury
Oct 2, 2019·Clinical and Experimental Emergency Medicine·Nicola ParentiGianfranco Cervellin
Dec 5, 2019·The Journal of the American Academy of Orthopaedic Surgeons·Joaquin PalmaJorge Filippi
Mar 7, 2019·The Angle Orthodontist·Sahar HaddadpourMahtab Nouri
Jan 31, 2015·The American Journal of Surgical Pathology·Kate O'ConnorDavid F Schaeffer
Jul 21, 2020·The Journal of Pathology·Adrian B LevineAli Bashashati
Feb 3, 2015·Academic Emergency Medicine : Official Journal of the Society for Academic Emergency Medicine·Henry E WangBrian Leroux
Jun 1, 2017·The Journal of Manual & Manipulative Therapy·Sean WillisShawn M Robbins
Jun 15, 2018·Journal of Experimental Orthopaedics·Malou E SlichterTom M Piscaer
Jul 16, 2019·Acute Medicine & Surgery·Yujiro MatsuishiYoshiaki Inoue
Oct 16, 2019·Revista colombiana de obstetricia y ginecología·Julie Angélica Mora-EnríquezMarcos Fidel Castillo-Zamora
Aug 7, 2020·Clinical Trials : Journal of the Society for Clinical Trials·Mengxi WangJose-Miguel Yamal
Feb 17, 2017·Clinical Orthopaedics and Related Research·Tomas ZamoraEduardo Botello
Jan 9, 2021·Archives of Orthopaedic and Trauma Surgery·Shinichi KuriyamaShuichi Matsuda
Nov 29, 2020·Journal of the American Medical Directors Association·Gustavo B LovadiniEdison I O Vidal
Feb 13, 2021·The Spine Journal : Official Journal of the North American Spine Society·Julio UrrutiaMarcelo Valacco
Feb 21, 2021·British Journal of Clinical Pharmacology·Johan LönnbroSusanna M Wallerstedt

❮ Previous
Next ❯

Related Concepts

Trending Feeds

COVID-19

Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

Blastomycosis

Blastomycosis fungal infections spread through inhaling Blastomyces dermatitidis spores. Discover the latest research on blastomycosis fungal infections here.

Nuclear Pore Complex in ALS/FTD

Alterations in nucleocytoplasmic transport, controlled by the nuclear pore complex, may be involved in the pathomechanism underlying multiple neurodegenerative diseases including Amyotrophic Lateral Sclerosis and Frontotemporal Dementia. Here is the latest research on the nuclear pore complex in ALS and FTD.

Applications of Molecular Barcoding

The concept of molecular barcoding is that each original DNA or RNA molecule is attached to a unique sequence barcode. Sequence reads having different barcodes represent different original molecules, while sequence reads having the same barcode are results of PCR duplication from one original molecule. Discover the latest research on molecular barcoding here.

Chronic Fatigue Syndrome

Chronic fatigue syndrome is a disease characterized by unexplained disabling fatigue; the pathology of which is incompletely understood. Discover the latest research on chronic fatigue syndrome here.

Evolution of Pluripotency

Pluripotency refers to the ability of a cell to develop into three primary germ cell layers of the embryo. This feed focuses on the mechanisms that underlie the evolution of pluripotency. Here is the latest research.

Position Effect Variegation

Position Effect Variagation occurs when a gene is inactivated due to its positioning near heterochromatic regions within a chromosome. Discover the latest research on Position Effect Variagation here.

STING Receptor Agonists

Stimulator of IFN genes (STING) are a group of transmembrane proteins that are involved in the induction of type I interferon that is important in the innate immune response. The stimulation of STING has been an active area of research in the treatment of cancer and infectious diseases. Here is the latest research on STING receptor agonists.

Microbicide

Microbicides are products that can be applied to vaginal or rectal mucosal surfaces with the goal of preventing, or at least significantly reducing, the transmission of sexually transmitted infections. Here is the latest research on microbicides.