Reader studies for validation of CAD systems

Neural Networks : the Official Journal of the International Neural Network Society
Brandon D Gallas, David G Brown

Abstract

Evaluation of computational intelligence (CI) systems designed to improve the performance of a human operator is complicated by the need to include the effect of human variability. In this paper we consider human (reader) variability in the context of medical imaging computer-assisted diagnosis (CAD) systems, and we outline how to compare the detection performance of readers with and without the CAD. An effective and statistically powerful comparison can be accomplished with a receiver operating characteristic (ROC) experiment, summarized by the reader-averaged area under the ROC curve (AUC). The comparison requires sophisticated yet well-developed methods for multi-reader multi-case (MRMC) variance analysis. MRMC variance analysis accounts for random readers, random cases, and correlations in the experiment. In this paper, we extend the methods available for estimating this variability. Specifically, we present a method that can treat arbitrary study designs. Most methods treat only the fully-crossed study design, where every reader reads every case in two experimental conditions. We demonstrate our method with a computer simulation, and we assess the statistical power of a variety of study designs.

References

Sep 1, 1986·Investigative Radiology·C E Metz
Jul 1, 1988·Medical Decision Making : an International Journal of the Society for Medical Decision Making·A N Tosteson, C B Begg
Aug 15, 1996·Statistics in Medicine·X H Zhou, C A Gatsonis
Mar 9, 1999·Journal of the Optical Society of America. A, Optics, Image Science, and Vision·A E Burgess
Mar 27, 2001·Journal of the Optical Society of America. A, Optics, Image Science, and Vision·C K Abbey, H H Barrett
Aug 23, 2003·Biostatistics·Todd A Alonzo, Margaret Sullivan Pepe
Apr 9, 2005·The New England Journal of Medicine·Robert Tibshirani
Sep 15, 2005·Statistics in Medicine·Timothy D Johnson, Valen E Johnson
Feb 21, 2006·Academic Radiology·Brandon D Gallas
Oct 27, 2006·IEEE Transactions on Pattern Analysis and Machine Intelligence·Waleed A YousefMurray H Loew
Oct 31, 2006·Academic Radiology·Eric ClarksonHarrison H Barrett
May 16, 2007·Academic Radiology·Robert F WagnerGregory Campbell
Dec 7, 2007·Journal of the Optical Society of America. A, Optics, Image Science, and Vision·Brandon D GallasKyle J Myers

❮ Previous
Next ❯

Citations

Aug 10, 2013·Medical Physics·Nicholas PetrickHeang-Ping Chan
Nov 6, 2012·Academic Radiology·Nancy A ObuchowskiStephen L Hillis
Jun 22, 2012·Academic Radiology·Weijie ChenBerkman Sahiner
Feb 7, 2012·Academic Radiology·Brandon D GallasMargarita L Zuley
Mar 6, 2019·Journal of the National Cancer Institute·Alejandro Rodriguez-RuizIoannis Sechopoulos
Jan 11, 2012·Revista Da Sociedade Brasileira De Medicina Tropical·Leonardo Ponce da MottaMarcelo Rosandiski Lyra
Feb 5, 2019·Journal of Medical Imaging·Brandon D GallasKyle J Myers
Sep 2, 2021·Statistical Methods in Medical Research·Erich P Huang, Joanna H Shih

❮ Previous
Next ❯

Related Concepts

Related Feeds

Brain Imaging of Neural Circuits

Neural circuits are groups of interconnected neurons which carry out specific functions when activated. Imaging these neural circuits allows researches to further elucidate their mechanisms and functions. Follow this feed to stay up to date on brain imaging of neural circuits.

Barrett Esophagus

Barrett’s esophagus if a serious complication of gastroesophageal reflux disease during which the normal esophageal lining changes to tissue that resembles intestinal lining. Here is the latest research.

Brain Imaging of Neural Circuits (MDS)

Neural circuits are groups of interconnected neurons which carry out specific functions when activated. Imaging these neural circuits allows researches to further elucidate their mechanisms and functions. Follow this feed to stay up to date on brain imaging of neural circuits.

Related Papers

Journal of the Optical Society of America. A, Optics, Image Science, and Vision
Brandon D GallasKyle J Myers
© 2021 Meta ULC. All rights reserved