Audio-visual integration in noise: Influence of auditory and visual stimulus degradation on eye movements and perception of the McGurk effect

Attention, Perception & Psychophysics
Jemaine E StaceyPaula C Stacey

Abstract

Seeing a talker's face can aid audiovisual (AV) integration when speech is presented in noise. However, few studies have simultaneously manipulated auditory and visual degradation. We aimed to establish how degrading the auditory and visual signal affected AV integration. Where people look on the face in this context is also of interest; Buchan, Paré and Munhall (Brain Research, 1242, 162-171, 2008) found fixations on the mouth increased in the presence of auditory noise whilst Wilson, Alsius, Paré and Munhall (Journal of Speech, Language, and Hearing Research, 59(4), 601-615, 2016) found mouth fixations decreased with decreasing visual resolution. In Condition 1, participants listened to clear speech, and in Condition 2, participants listened to vocoded speech designed to simulate the information provided by a cochlear implant. Speech was presented in three levels of auditory noise and three levels of visual blurring. Adding noise to the auditory signal increased McGurk responses, while blurring the visual signal decreased McGurk responses. Participants fixated the mouth more on trials when the McGurk effect was perceived. Adding auditory noise led to people fixating the mouth more, while visual degradation led to people fixat...Continue Reading

References

Dec 23, 1976·Nature·H McGurk, J MacDonald
Aug 1, 1990·Hearing Research·B R Glasberg, B C Moore
Feb 1, 1990·British Journal of Audiology·S Gatehouse, J Gordon
Oct 13, 1995·Science·R V ShannonM Ekelid
Aug 1, 1995·Perception & Psychophysics·J E Hoffman, B Subramaniam
Jan 1, 1997·Perception·C S Campbell, D W Massaro
Aug 27, 1998·Perception & Psychophysics·E Vatikiotis-BatesonK G Munhall
Sep 24, 1999·The Journal of the Acoustical Society of America·K W Grant, P F Seitz
Dec 29, 2000·Nature·L ShamsS Shimojo
Feb 28, 2001·Perception·J MacDonaldT Bachmann
Aug 15, 2001·Current Opinion in Neurobiology·S Shimojo, L Shams
Sep 25, 2002·Perception & Psychophysics·Sharon M Thomas, Timothy R Jordan
Jul 26, 2003·The Journal of the Acoustical Society of America·Michael K Qin, Andrew J Oxenham
Mar 31, 2004·Trends in Cognitive Sciences·Marc O Ernst, Heinrich H Bülthoff
Aug 18, 2004·Perception & Psychophysics·K G MunhallE Vatikiotis-Bateson
Oct 30, 2004·Brain Research. Cognitive Brain Research·Tobias S AndersenMikko Sams
Nov 5, 2005·Neuron·Ilana B Witten, Eric I Knudsen
Apr 4, 2007·Proceedings of the National Academy of Sciences of the United States of America·J RougerP Barone
May 8, 2010·Physics of Life Reviews·Ladan Shams, Robyn Kim
May 18, 2010·Ear and Hearing·Nancy Tye-MurraySandra Hale
Jan 24, 2012·Neuropsychologia·Carolyn McGettiganSophie K Scott
Jan 10, 2014·Journal of Memory and Language·Dale J BarrHarry J Tily
Dec 19, 2014·Frontiers in Neuroscience·Lynne E Bernstein, Einat Liebenthal
Mar 25, 2015·Psychonomic Bulletin & Review·Debshila Basu MallickMichael S Beauchamp
Mar 27, 2015·Attention, Perception & Psychophysics·Demet GurlerMichael Beauchamp
Feb 3, 2016·Attention, Perception & Psychophysics·Christopher W RobinsonScott Sinnett
May 7, 2016·Attention, Perception & Psychophysics·Agnès AlsiusKevin G Munhall
Aug 19, 2016·Journal of Speech, Language, and Hearing Research : JSLHR·Amanda H WilsonKevin G Munhall
Aug 28, 2016·Journal of Speech, Language, and Hearing Research : JSLHR·Nancy Tye-MurrayJoe Barcroft
Dec 7, 2016·Attention, Perception & Psychophysics·Kristin J Van EngenBharath Chandrasekaran
Jul 7, 2017·Frontiers in Psychology·Eliana MastrantuonoIsabel R Rodríguez-Ortiz
Aug 19, 2018·Scientific Reports·Rebecca J HirstHarriet A Allen
Jan 1, 2018·Multisensory Research·Agnès AlsiusKevin G Munhall

❮ Previous
Next ❯

Citations

Nov 23, 2020·Cortex; a Journal Devoted to the Study of the Nervous System and Behavior·John F MagnottiMichael S Beauchamp
May 15, 2021·Cortex; a Journal Devoted to the Study of the Nervous System and Behavior·Kaoru SekiyamaRyoko Mugitani
Sep 1, 2021·Journal of Speech, Language, and Hearing Research : JSLHR·Briony BanksPatti Adank
Dec 3, 2021·Psychological Research·Basil WahnAnne Böckler-Raettig

❮ Previous
Next ❯

Methods Mentioned

BETA
cochlear implants

Software Mentioned

Matlab
R
lmerControl
Premiere Pro
lmer
Adobe Premiere Pro

Related Concepts

Related Feeds

Auditory Perception

Auditory perception is the ability to receive and interpret information attained by the ears. Here is the latest research on factors and underlying mechanisms that influence auditory perception.