Applying Kane's validity framework to a simulation based assessment of clinical competence

Advances in Health Sciences Education : Theory and Practice
Walter TavaresMaud Huiskamp

Abstract

Assessment of clinical competence is complex and inference based. Trustworthy and defensible assessment processes must have favourable evidence of validity, particularly where decisions are considered high stakes. We aimed to organize, collect and interpret validity evidence for a high stakes simulation based assessment strategy for certifying paramedics, using Kane's validity framework, which some report as challenging to implement. We describe our experience using the framework, identifying challenges, decisions points, interpretations and lessons learned. We considered data related to four inferences (scoring, generalization, extrapolation, implications) occurring during assessment and treated validity as a series of assumptions we must evaluate, resulting in several hypotheses and proposed analyses. We then interpreted our findings across the four inferences, judging if the evidence supported or refuted our proposed uses of the assessment data. Data evaluating "Scoring" included: (a) desirable tool characteristics, with acceptable inter-item correlations (b) strong item-total correlations (c) low error variance for items and raters, and (d) strong inter-rater reliability. Data evaluating "Generalizability" included: (a) a r...Continue Reading

References

Jul 13, 2002·Academic Medicine : Journal of the Association of American Medical Colleges·Susan Humphrey-Murto, John C MacFadyen
Jan 23, 2010·Advances in Health Sciences Education : Theory and Practice·Brian E ClauserRichard E Hawkins
Aug 19, 2011·Medical Education·Maria Mylopoulos, Glenn Regehr
Jul 28, 2012·Prehospital Emergency Care : Official Journal of the National Association of EMS Physicians and the National Association of State EMS Directors·Walter TavaresKevin W Eva
May 3, 2013·Advances in Health Sciences Education : Theory and Practice·David A CookRyan Brydges
Aug 22, 2013·Prehospital Emergency Care : Official Journal of the National Association of EMS Physicians and the National Association of State EMS Directors·Walter TavaresKevin W Eva
Feb 24, 2015·Advances in Health Sciences Education : Theory and Practice·Rose HatalaRichard Hawkins
May 20, 2015·Medical Education·David A CookRose Hatala
Oct 25, 2016·American Journal of Surgery·Alicia Ponton-CarssIrene W Y Ma
Sep 9, 2016·BMC Health Services Research·Walter TavaresBecky Donelon
Oct 4, 2016·Advances in Health Sciences Education : Theory and Practice·Christina St-OngeBrian Hodges

❮ Previous
Next ❯

Citations

Dec 19, 2019·Journal of Evaluation in Clinical Practice·Rylan EganBrent Thoma
Mar 26, 2019·Plastic and Reconstructive Surgery·Christine FahimRanil R Sonnadara
Oct 26, 2018·Perspectives on Medical Education·Dilshan Pieris
Apr 24, 2020·Journal of Graduate Medical Education·Caroline AndlerSandrijn M van Schaik
Sep 4, 2020·Medical Education·Walter TavaresMaud Huiskamp
Jun 21, 2019·Pharmacy : Journal of Pharmacy, Education and Practice·Hayley CroftJennifer Schneider
Nov 16, 2018·Perspectives on Medical Education·Harold G J BokKent G Hecker
Oct 22, 2020·Canadian Journal of Anaesthesia = Journal Canadien D'anesthésie·Daniel G DuboisMeghan M McConnell
Mar 30, 2021·BJA Education·J Dupre, V N Naik
Aug 10, 2021·Advances in Health Sciences Education : Theory and Practice·Walter TavaresRyan Brydges
Jun 9, 2019·Journal of Surgical Education·Sydney McQueenRanil Sonnadara

❮ Previous
Next ❯

Related Concepts

Trending Feeds

COVID-19

Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

Blastomycosis

Blastomycosis fungal infections spread through inhaling Blastomyces dermatitidis spores. Discover the latest research on blastomycosis fungal infections here.

Nuclear Pore Complex in ALS/FTD

Alterations in nucleocytoplasmic transport, controlled by the nuclear pore complex, may be involved in the pathomechanism underlying multiple neurodegenerative diseases including Amyotrophic Lateral Sclerosis and Frontotemporal Dementia. Here is the latest research on the nuclear pore complex in ALS and FTD.

Applications of Molecular Barcoding

The concept of molecular barcoding is that each original DNA or RNA molecule is attached to a unique sequence barcode. Sequence reads having different barcodes represent different original molecules, while sequence reads having the same barcode are results of PCR duplication from one original molecule. Discover the latest research on molecular barcoding here.

Chronic Fatigue Syndrome

Chronic fatigue syndrome is a disease characterized by unexplained disabling fatigue; the pathology of which is incompletely understood. Discover the latest research on chronic fatigue syndrome here.

Evolution of Pluripotency

Pluripotency refers to the ability of a cell to develop into three primary germ cell layers of the embryo. This feed focuses on the mechanisms that underlie the evolution of pluripotency. Here is the latest research.

Position Effect Variegation

Position Effect Variagation occurs when a gene is inactivated due to its positioning near heterochromatic regions within a chromosome. Discover the latest research on Position Effect Variagation here.

STING Receptor Agonists

Stimulator of IFN genes (STING) are a group of transmembrane proteins that are involved in the induction of type I interferon that is important in the innate immune response. The stimulation of STING has been an active area of research in the treatment of cancer and infectious diseases. Here is the latest research on STING receptor agonists.

Microbicide

Microbicides are products that can be applied to vaginal or rectal mucosal surfaces with the goal of preventing, or at least significantly reducing, the transmission of sexually transmitted infections. Here is the latest research on microbicides.