Connecting Deep Neural Networks to Physical, Perceptual, and Electrophysiological Auditory Signals

Frontiers in Neuroscience
Nicholas HuangMounya Elhilali

Abstract

Deep neural networks have been recently shown to capture intricate information transformation of signals from the sensory profiles to semantic representations that facilitate recognition or discrimination of complex stimuli. In this vein, convolutional neural networks (CNNs) have been used very successfully in image and audio classification. Designed to imitate the hierarchical structure of the nervous system, CNNs reflect activation with increasing degrees of complexity that transform the incoming signal onto object-level representations. In this work, we employ a CNN trained for large-scale audio object classification to gain insights about the contribution of various audio representations that guide sound perception. The analysis contrasts activation of different layers of a CNN with acoustic features extracted directly from the scenes, perceptual salience obtained from behavioral responses of human listeners, as well as neural oscillations recorded by electroencephalography (EEG) in response to the same natural scenes. All three measures are tightly linked quantities believed to guide percepts of salience and object formation when listening to complex scenes. The results paint a picture of the intricate interplay between lo...Continue Reading

References

Jun 15, 2000·Proceedings of the National Academy of Sciences of the United States of America·C HaenschelJ Gruzelier
Dec 5, 2000·International Journal of Psychophysiology : Official Journal of the International Organization of Psychophysiology·O Bertrand, C Tallon-Baudry
Dec 5, 2000·International Journal of Psychophysiology : Official Journal of the International Organization of Psychophysiology·M M MüllerA Keil
Sep 11, 2003·Current Opinion in Neurobiology·Stefan Treue
May 21, 2004·Nature Reviews. Neuroscience·Jeremy M Wolfe, Todd S Horowitz
Nov 8, 2005·Current Biology : CB·Christoph KayserNikos K Logothetis
Jun 9, 2007·Cerebral Cortex·Sam M DoesburgLawrence M Ward
Nov 8, 2008·Science·Elia FormisanoRainer Goebel
Mar 10, 2009·Current Biology : CB·Noël StaerenElia Formisano
Jan 22, 2011·Computational Intelligence and Neuroscience·Robert OostenveldJan-Mathijs Schoffelen
Nov 13, 2012·PloS One·Anatole NöstlPatrik Sörqvist
May 9, 2014·Proceedings of the National Academy of Sciences of the United States of America·Daniel L K YaminsJames J DiCarlo
Jun 7, 2014·Frontiers in Human Neuroscience·Emine Merve Kaya, Mounya Elhilali
Aug 8, 2014·Frontiers in Neuroscience·Lan Shuai, Mounya Elhilali
Jul 15, 2015·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Umut Güçlü, Marcel A J van Gerven
Jul 21, 2015·Current Biology : CB·Luc H ArnalDavid Poeppel
Feb 26, 2016·Nature Neuroscience·Daniel L K Yamins, James J DiCarlo
Oct 31, 2016·Hearing Research·Theofilos PetsasMaria Chait
Jan 4, 2017·Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences·Richard VealeMasatoshi Yoshida
Apr 5, 2017·The Journal of the Acoustical Society of America·Nicholas Huang, Mounya Elhilali
Nov 24, 2015·Annual Review of Vision Science·Nikolaus Kriegeskorte

❮ Previous
Next ❯

Citations

Jun 25, 2020·Apoptosis : an International Journal on Programmed Cell Death·L L M LuijerinkR Machaalani
Mar 7, 2021·Ear and Hearing·Jan-Willem A WasmannDennis L Barbour
Jan 29, 2022·Nature Human Behaviour·Rufin VanRullen

❮ Previous
Next ❯

Software Mentioned

EEGLab
FieldTrip
FreeSound
Cleanline MATLAB

Related Concepts

Related Feeds

Auditory Perception

Auditory perception is the ability to receive and interpret information attained by the ears. Here is the latest research on factors and underlying mechanisms that influence auditory perception.