Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants

Behavior Research Methods
David J Hauser, Norbert Schwarz

Abstract

Participant attentiveness is a concern for many researchers using Amazon's Mechanical Turk (MTurk). Although studies comparing the attentiveness of participants on MTurk versus traditional subject pool samples have provided mixed support for this concern, attention check questions and other methods of ensuring participant attention have become prolific in MTurk studies. Because MTurk is a population that learns, we hypothesized that MTurkers would be more attentive to instructions than are traditional subject pool samples. In three online studies, participants from MTurk and collegiate populations participated in a task that included a measure of attentiveness to instructions (an instructional manipulation check: IMC). In all studies, MTurkers were more attentive to the instructions than were college students, even on novel IMCs (Studies 2 and 3), and MTurkers showed larger effects in response to a minute text manipulation. These results have implications for the sustainable use of MTurk samples for social science research and for the conclusions drawn from research with MTurk and college subject pool samples.

References

May 27, 2004·Journal of Personality and Social Psychology·Yaacov SchulEugene Burnstein
Jul 1, 2011·Behavior Research Methods·Winter Mason, Siddharth Suri
Mar 22, 2013·PloS One·Matthew J C CrumpTodd M Gureckis
Jul 10, 2013·Behavior Research Methods·Jesse ChandlerGabriele Paolacci
Dec 21, 2013·Behavior Research Methods·Eyal PeerAlessandro Acquisti

❮ Previous
Next ❯

Citations

Jan 17, 2016·Annual Review of Clinical Psychology·Jesse Chandler, Danielle Shapiro
May 18, 2016·JMIR Mental Health·Scott R BraithwaiteCarl Lee Hanson
Aug 6, 2016·Frontiers in Psychology·Matthew L TompkinsAnne M Aimola Davies
Sep 3, 2016·Addictive Behaviors·Heather KriegerClayton Neighbors
Nov 15, 2016·The Journal of Urology·Michael L Garcia-RoigAndrew J Kirsch
Oct 4, 2016·Cognitive Behaviour Therapy·Nicholas P AllanNorman B Schmidt
Nov 30, 2016·Social Psychiatry and Psychiatric Epidemiology·Magen Seymour-SmithWendy Brodribb
Apr 1, 2017·Attention, Perception & Psychophysics·Mackenzie A SundayIsabel Gauthier
Nov 30, 2016·Science and Engineering Ethics·Andrea Polonioli
Jan 23, 2017·American Journal of Preventive Medicine·Cindy W LeungEric B Rimm
Sep 29, 2017·International Journal of Environmental Research and Public Health·Linda FleisherFlaura Winston
Sep 6, 2017·Psychological Reports·Marisa K Crowder, Markus Kemmelmeier
Dec 9, 2017·Frontiers in Psychology·Magnus BergquistAndré Hansla
May 17, 2018·Journal of Applied Animal Welfare Science : JAAWS·Natalie YoungGigi Davidson
Nov 29, 2017·Evolutionary Psychology : an International Journal of Evolutionary Approaches to Psychology and Behavior·Jinguang Zhang
Oct 4, 2017·Perspectives on Psychological Science : a Journal of the Association for Psychological Science·Patricia M Greenfield
Dec 16, 2017·Journal of Gambling Studies·Billy A PalmerM Kristina DePue
Oct 19, 2017·Psychological Medicine·N J Raihani, V Bell
Jun 30, 2018·PloS One·Olaf Strelcyk, Gurjit Singh
Jul 11, 2018·Depression and Anxiety·Lisa Hancock, Richard A Bryant
Jul 14, 2018·European Eating Disorders Review : the Journal of the Eating Disorders Association·Ashley A WiedemannCarlos M Grilo
Dec 7, 2017·Culture, Health & Sexuality·Brian D EarpWilliam A Jellison
Aug 10, 2018·PloS One·Marta WalentynowiczArthur A Stone
Sep 19, 2018·Obesity·Christina M Hopkins, Gary G Bennett
Jan 25, 2018·Journal of Clinical Child and Adolescent Psychology : the Official Journal for the Society of Clinical Child and Adolescent Psychology, American Psychological Association, Division 53·Jessica L Schleider, John R Weisz
Feb 8, 2019·Suicide & Life-threatening Behavior·Alejandro TrujilloApril R Smith
May 28, 2019·Nicotine & Tobacco Research : Official Journal of the Society for Research on Nicotine and Tobacco·Jessica L KingErin L Sutfin
May 15, 2018·Assessment·Peter J CastagnaMatthew Calamia

❮ Previous
Next ❯

Related Concepts

Trending Feeds

COVID-19

Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

Blastomycosis

Blastomycosis fungal infections spread through inhaling Blastomyces dermatitidis spores. Discover the latest research on blastomycosis fungal infections here.

Nuclear Pore Complex in ALS/FTD

Alterations in nucleocytoplasmic transport, controlled by the nuclear pore complex, may be involved in the pathomechanism underlying multiple neurodegenerative diseases including Amyotrophic Lateral Sclerosis and Frontotemporal Dementia. Here is the latest research on the nuclear pore complex in ALS and FTD.

Applications of Molecular Barcoding

The concept of molecular barcoding is that each original DNA or RNA molecule is attached to a unique sequence barcode. Sequence reads having different barcodes represent different original molecules, while sequence reads having the same barcode are results of PCR duplication from one original molecule. Discover the latest research on molecular barcoding here.

Chronic Fatigue Syndrome

Chronic fatigue syndrome is a disease characterized by unexplained disabling fatigue; the pathology of which is incompletely understood. Discover the latest research on chronic fatigue syndrome here.

Evolution of Pluripotency

Pluripotency refers to the ability of a cell to develop into three primary germ cell layers of the embryo. This feed focuses on the mechanisms that underlie the evolution of pluripotency. Here is the latest research.

Position Effect Variegation

Position Effect Variagation occurs when a gene is inactivated due to its positioning near heterochromatic regions within a chromosome. Discover the latest research on Position Effect Variagation here.

STING Receptor Agonists

Stimulator of IFN genes (STING) are a group of transmembrane proteins that are involved in the induction of type I interferon that is important in the innate immune response. The stimulation of STING has been an active area of research in the treatment of cancer and infectious diseases. Here is the latest research on STING receptor agonists.

Microbicide

Microbicides are products that can be applied to vaginal or rectal mucosal surfaces with the goal of preventing, or at least significantly reducing, the transmission of sexually transmitted infections. Here is the latest research on microbicides.

Related Papers

Behavior Research Methods
Amy Summerville, Christopher R Chartier
Drug and Alcohol Review
Marcella H Boynton, Laura Smart Richman
Journal of Attention Disorders
Brian T Wymbs, Anne E Dawson
© 2021 Meta ULC. All rights reserved