Limitations of Standard Accessible Captioning of Sounds and Music for Deaf and Hard of Hearing People: An EEG Study

Frontiers in Integrative Neuroscience
Pablo RevueltaJosé Manuel Sánchez-Pena

Abstract

Captioning is the process of transcribing speech and acoustical information into text to help deaf and hard of hearing people accessing to the auditory track of audiovisual media. In addition to the verbal transcription, it includes information such as sound effects, speaker identification, or music tagging. However, it just takes into account a limited spectrum of the whole acoustic information available in the soundtrack, and hence, an important amount of emotional information is lost when attending just to the normative compliant captions. In this article, it is shown, by means of behavioral and EEG measurements, how emotional information related to sounds and music used by the creator in the audiovisual work is perceived differently by normal hearing group and hearing disabled group when applying standard captioning. Audio and captions activate similar processing areas, respectively, in each group, although not with the same intensity. Moreover, captions require higher activation of voluntary attentional circuits, as well as language-related areas. Captions transcribing musical information increase attentional activity, instead of emotional processing.

References

Jul 1, 1986·Brain and Cognition·E K Silberman, H Weingartner
Aug 1, 1980·Electroencephalography and Clinical Neurophysiology·H ShibasakiA M Halliday
Nov 18, 2000·Journal of Personality and Social Psychology·M C Green, T C Brock
Dec 18, 2001·International Journal of Psychophysiology : Official Journal of the International Organization of Psychophysiology·C H Brunia, G J van Boxtel
Dec 18, 2001·International Journal of Psychophysiology : Official Journal of the International Organization of Psychophysiology·K B BöckerM N Verbaten
Aug 5, 2006·Science·Benedetto De MartinoRaymond J Dolan
Mar 30, 2007·Brain : a Journal of Neurology·Ingrid R OlsonYoussef Ezzyat
Jan 1, 2008·Biological Psychology·Jonas K OlofssonJohn Polich
Oct 3, 2009·Clinical Neurophysiology : Official Journal of the International Federation of Clinical Neurophysiology·Connie C DuncanCyma Van Petten
Feb 11, 2011·Brain and Cognition·Marta Bianchin, Alessandro Angrilli
Sep 6, 2011·Neuropsychologia·Laura M SkipperIngrid R Olson
Jun 20, 2012·Journal of Experimental Psychology. Human Perception and Performance·Frank A RussoDeborah I Fels
Jul 4, 2012·Frontiers in Human Neuroscience·Ulrike AltmannArthur M Jacobs
May 21, 2013·Social Cognitive and Affective Neuroscience·Sabine AustMalek Bajbouj
Aug 7, 2013·Journal of Cognitive Neuroscience·Thomas A CarlsonL Robert Slevc
Dec 4, 2013·Social Cognitive and Affective Neuroscience·Corinna PehrsLars Kuchinke
Nov 26, 2015·Cerebral Cortex·Corinna PehrsStefan Koelsch
Mar 24, 2016·Cognitive, Affective & Behavioral Neuroscience·Eva M MorenoManuel Martín-Loeches
Mar 30, 2016·Neuropsychologia·Carlos Romero-RivasAlbert Costa

❮ Previous
Next ❯

Software Mentioned

Phone
Applause

Related Concepts

Related Feeds

Brain Circuits in Emotional Learning

The neuronal circuits within the cortico-limbic brain regions form networks that mediate emotional behavior. Areas specific to emotional learning include the basal amygdala and sublenticular extended amygdala region along with a supplemental motor area. Discover the latest research on brain circuits in emotional learning here.

Auditory Perception

Auditory perception is the ability to receive and interpret information attained by the ears. Here is the latest research on factors and underlying mechanisms that influence auditory perception.