Cortical network responses map onto data-driven features that capture visual semantics of movie fragments.

Scientific Reports
Julia BerezutskayaNick F Ramsey

Abstract

Research on how the human brain extracts meaning from sensory input relies in principle on methodological reductionism. In the present study, we adopt a more holistic approach by modeling the cortical responses to semantic information that was extracted from the visual stream of a feature film, employing artificial neural network models. Advances in both computer vision and natural language processing were utilized to extract the semantic representations from the film by combining perceptual and linguistic information. We tested whether these representations were useful in studying the human brain data. To this end, we collected electrocorticography responses to a short movie from 37 subjects and fitted their cortical patterns across multiple regions using the semantic components extracted from film frames. We found that individual semantic components reflected fundamental semantic distinctions in the visual input, such as presence or absence of people, human movement, landscape scenes, human faces, etc. Moreover, each semantic component mapped onto a distinct functional cortical network involving high-level cognitive regions in occipitotemporal, frontal and parietal cortices. The present work demonstrates the potential of the ...Continue Reading

References

Jan 1, 1992·Trends in Neurosciences·M A Goodale, A D Milner
Apr 29, 1998·Nature·R Epstein, N Kanwisher
Apr 27, 2001·Vision Research·K Grill-SpectorN Kanwisher
Feb 3, 2004·Human Brain Mapping·Andreas Bartels, Semir Zeki
Jun 26, 2004·Annual Review of Neuroscience·Kalanit Grill-Spector, Rafael Malach
Apr 15, 2005·Journal of Cognitive Neuroscience·Alice J O'TooleJames V Haxby
Apr 16, 2005·Current Opinion in Neurobiology·Michael S Beauchamp
Apr 16, 2005·Current Opinion in Neurobiology·Luciano FadigaEtienne Olivier
Nov 25, 2005·Nature Neuroscience·Nicole C Rust, J Anthony Movshon
Sep 14, 2006·Annual Review of Psychology·Alex Martin
Jan 16, 2007·Science·Brendan J Frey, Delbert Dueck
Mar 7, 2008·Nature·Kendrick N KayJack L Gallant
May 31, 2008·Science·Tom M MitchellMarcel Adam Just
Nov 7, 2008·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Supratim RaySteven S Hsiao
Dec 27, 2008·Neuron·Nikolaus KriegeskortePeter A Bandettini
Oct 20, 2009·Journal of Neuroscience Methods·Dora HermesNick F Ramsey
Jan 16, 2010·The European Journal of Neuroscience·Juliana Dushanova, John Donoghue
Aug 11, 2010·Journal of Comparative Psychology·David A Leopold, Gillian Rhodes
Mar 11, 2011·Human Brain Mapping·Marie-Hélène GrosbrasSimon B Eickhoff
Mar 19, 2011·Nature Reviews. Neuroscience·Dwight J KravitzMortimer Mishkin
Jun 22, 2011·Human Brain Mapping·Dora HermesNick F Ramsey
Feb 24, 2012·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Andrew C ConnollyJames V Haxby
Mar 8, 2012·Neural Networks : the Official Journal of the International Neural Network Society·J StallkampC Igel
Apr 5, 2012·Journal of Physiology, Paris·Thomas NaselarisJack L Gallant
Nov 9, 2012·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Marius V Peelen, Alfonso Caramazza
Jan 25, 2013·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Daniel D DilksNancy Kanwisher
Jan 28, 2014·Nature Neuroscience·Radoslaw Martin CichyAude Oliva
Apr 20, 2014·PLoS Computational Biology·Hamed NiliNikolaus Kriegeskorte
Jun 26, 2014·Nature Reviews. Neuroscience·Kalanit Grill-Spector, Kevin S Weiner
Aug 8, 2014·PLoS Computational Biology·Umut Güçlü, Marcel A J van Gerven
Nov 7, 2014·PLoS Computational Biology·Seyed-Mahdi Khaligh-Razavi, Nikolaus Kriegeskorte
Dec 19, 2014·PLoS Computational Biology·Charles F CadieuJames J DiCarlo
Jul 15, 2015·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Umut Güçlü, Marcel A J van Gerven
Jul 28, 2015·Neuropsychologia·Corentin JacquesKalanit Grill-Spector
Aug 12, 2015·Wiley Interdisciplinary Reviews. Cognitive Science·B de GelderR Watson
Feb 26, 2016·Nature Neuroscience·Daniel L K Yamins, James J DiCarlo

❮ Previous
Next ❯

Software Mentioned

fastText
learn
scikit
MATLAB
SPM8
Freesurfer
Freeview
Presentation
Clarifai

Related Concepts

Trending Feeds

COVID-19

Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

Blastomycosis

Blastomycosis fungal infections spread through inhaling Blastomyces dermatitidis spores. Discover the latest research on blastomycosis fungal infections here.

Nuclear Pore Complex in ALS/FTD

Alterations in nucleocytoplasmic transport, controlled by the nuclear pore complex, may be involved in the pathomechanism underlying multiple neurodegenerative diseases including Amyotrophic Lateral Sclerosis and Frontotemporal Dementia. Here is the latest research on the nuclear pore complex in ALS and FTD.

Applications of Molecular Barcoding

The concept of molecular barcoding is that each original DNA or RNA molecule is attached to a unique sequence barcode. Sequence reads having different barcodes represent different original molecules, while sequence reads having the same barcode are results of PCR duplication from one original molecule. Discover the latest research on molecular barcoding here.

Chronic Fatigue Syndrome

Chronic fatigue syndrome is a disease characterized by unexplained disabling fatigue; the pathology of which is incompletely understood. Discover the latest research on chronic fatigue syndrome here.

Evolution of Pluripotency

Pluripotency refers to the ability of a cell to develop into three primary germ cell layers of the embryo. This feed focuses on the mechanisms that underlie the evolution of pluripotency. Here is the latest research.

Position Effect Variegation

Position Effect Variagation occurs when a gene is inactivated due to its positioning near heterochromatic regions within a chromosome. Discover the latest research on Position Effect Variagation here.

STING Receptor Agonists

Stimulator of IFN genes (STING) are a group of transmembrane proteins that are involved in the induction of type I interferon that is important in the innate immune response. The stimulation of STING has been an active area of research in the treatment of cancer and infectious diseases. Here is the latest research on STING receptor agonists.

Microbicide

Microbicides are products that can be applied to vaginal or rectal mucosal surfaces with the goal of preventing, or at least significantly reducing, the transmission of sexually transmitted infections. Here is the latest research on microbicides.