Using goal-driven deep learning models to understand sensory cortex

Nature Neuroscience
Daniel L K Yamins, James J DiCarlo


Fueled by innovation in the computer vision and artificial intelligence communities, recent developments in computational neuroscience have used goal-driven hierarchical convolutional neural networks (HCNNs) to make strides in modeling neural single-unit and population responses in higher visual cortical areas. In this Perspective, we review the recent progress in a broader modeling context and describe some of the key technical innovations that have supported it. We then outline how the goal-driven HCNN approach can be used to delve even more deeply into understanding the development and organization of sensory cortical processing.


Jun 14, 2016·ELife·Danique JeurissenPieter R Roelfsema
Sep 8, 2016·Frontiers in Synaptic Neuroscience·Tatyana O Sharpee
Aug 31, 2016·Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences·Nikolaus Kriegeskorte, Jörn Diedrichsen
Sep 30, 2016·Frontiers in Computational Neuroscience·Adam H MarblestoneKonrad P Kording
Feb 24, 2017·Nature Neuroscience·Jonathan D CohenTheodore L Willke
Apr 6, 2017·Neuropsychologia·Marius V Peelen, Paul E Downing
Jun 9, 2017·Nature Communications·Ryan J Rowekamp, Tatyana O Sharpee
Jun 14, 2017·Human Brain Mapping·Kun-Han LuZhongming Liu
Dec 10, 2016·PLoS Computational Biology·Benjamin R CowleyByron M Yu
Jan 20, 2017·ELife·Olivia Guest, Bradley C Love
Apr 24, 2018·PLoS Computational Biology·Michael F Bonner, Russell A Epstein
Nov 24, 2017·Scientific Reports·Keisuke SuzukiAnil K Seth
Dec 6, 2017·ELife·Jordan GuerguievBlake A Richards
Jun 6, 2018·Journal of Computational Neuroscience·Yimeng ZhangShiming Tang
Jun 1, 2018·ELife·Lotem Elber-Dorozko, Yonatan Loewenstein
Feb 7, 2018·Frontiers in Computational Neuroscience·Qiulei DongZhanyi Hu
Apr 6, 2018·Annual Review of Neuroscience·Rajeev V RikhyeMichael M Halassa
Jul 25, 2018·PLoS Computational Biology·Michelle R Greene, Bruce C Hansen
Jul 28, 2018·Annual Review of Vision Science·Andrea TacchettiTomaso A Poggio
Feb 3, 2018·Nature Communications·Devika NarainMehrdad Jazayeri
Feb 13, 2019·Scientific Data·Tomoyasu HorikawaYukiyasu Kamitani
Mar 19, 2019·PLoS Computational Biology·Christopher DiMattina, Curtis L Baker
Mar 21, 2019·Nature Communications·Katharina DobsNancy Kanwisher
Apr 24, 2019·PLoS Computational Biology·Santiago A CadenaAlexander S Ecker
Apr 27, 2019·Journal of Vision·William F KindelJoel Zylberberg
Feb 26, 2016·Nature Neuroscience·Anne K Churchland, L F Abbott
May 7, 2019·Glia·Ksenia V KastanenkaElena Galea
May 3, 2019·Science·Pouya BashivanJames J DiCarlo
Feb 17, 2018·Nature Reviews. Neuroscience·Pieter R Roelfsema, Anthony Holtmaat
Jun 13, 2018·Scientific Reports·Zohreh Gholami DoborjehAlexander Sumich
Jun 25, 2019·Human Brain Mapping·Ying YangElissa M Aminoff
Sep 17, 2019·Neural Computation·Luis Gonzalo Sánchez Giraldo, Odelia Schwartz
Aug 7, 2019·Annual Review of Vision Science·Daniel A Butts
Jan 30, 2018·Frontiers in Computational Neuroscience·Marcel A J van Gerven
Feb 7, 2020·Journal of Cognitive Neuroscience·Grace Lindsay
Dec 10, 2019·Human Brain Mapping·Xiaoxiao WangBensheng Qiu
Jan 15, 2019·PLoS Computational Biology·Guohua ShenYukiyasu Kamitani
Jun 24, 2020·Journal of Cognitive Neuroscience·John M HendersonTaylor R Hayes
Jul 3, 2020·PLoS Computational Biology·Julia BerezutskayaNick F Ramsey
Jul 22, 2020·Journal of Vision·Md Nasir Uddin LaskarOdelia Schwartz
Sep 3, 2020·ELife·Marin DujmovićJeffrey S Bowers
Oct 3, 2020·PLoS Computational Biology·Courtney J SpoererNikolaus Kriegeskorte
Aug 24, 2017·Scientific Reports·Devavrat VartakPieter R Roelfsema
Oct 24, 2017·Frontiers in Neuroscience·Danilo Bzdok


Jun 1, 1979·The Journal of Physiology·K K De ValoisE W Yund
Oct 1, 1978·The Journal of Physiology·J Anthony MovshonDavid J Tolhurst
Jan 1, 1991·Cerebral Cortex·D J Felleman, D C Van Essen
Dec 1, 1984·Cognition·S Ullman
May 1, 1995·Current Biology : CB·N K LogothetisTomaso A Poggio
Jan 1, 1995·Annual Review of Neuroscience·W Singer, C M Gray
May 26, 1995·Science·G E HintonR M Neal
Oct 1, 1996·Journal of Neurophysiology·Jack L GallantD C Van Essen
Oct 20, 1999·Nature Neuroscience·M Riesenhuber, Tomaso A Poggio
Jun 9, 2000·Annual Review of Neuroscience·R C deCharms, A Zador
Mar 26, 2002·Trends in Cognitive Sciences·Rafael MalachUri Hasson
Nov 12, 2002·Nature Neuroscience·Anitha Pasupathy, Charles E Connor
Jul 1, 1961·The Journal of Pathology and Bacteriology·M K ALEXANDER
Oct 1, 1959·The Journal of Physiology·D H HUBEL, T N WIESEL
Sep 6, 2005·Proceedings of the National Academy of Sciences of the United States of America·Valery A KalatskyMichael P Stryker
Nov 8, 2005·Science·Chou P HungJames J DiCarlo
Nov 18, 2005·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Matteo CarandiniNicole C Rust
Mar 7, 2006·Journal of Computational Neuroscience·David J KleinShihab A Shamma
Sep 22, 2006·Journal of Neurophysiology·Stephen V DavidJack L Gallant
Oct 17, 2006·Nature Neuroscience·Nicole C RustJ Anthony Movshon
Apr 4, 2007·Proceedings of the National Academy of Sciences of the United States of America·Thomas SerreTomaso A Poggio
Jul 17, 2007·Trends in Cognitive Sciences·James J DiCarlo, David D Cox
Nov 9, 2007·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Davide ZoccolanJames J DiCarlo
Jan 30, 2008·PLoS Computational Biology·Nicolas PintoJames J DiCarlo
Oct 7, 2008·Nature Neuroscience·Yukako YamaneCharles E Connor
Jan 1, 2009·Frontiers in Neuroscience·Nikolaus Kriegeskorte
Oct 1, 2010·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Nicole C Rust, James J DiCarlo
Apr 5, 2011·Neural Networks : the Official Journal of the International Neural Network Society·Xingjian Jing
Aug 16, 2011·Nature Neuroscience·Jeremy Freeman, Eero P Simoncelli
Nov 24, 2011·Nature Reviews. Neuroscience·Matteo Carandini, David J Heeger
Feb 3, 2012·Cerebral Cortex·Jeffrey M YauCharles E Connor
Feb 14, 2012·Neuron·James J DiCarloNicole C Rust
Dec 14, 2012·Nature Reviews. Cancer·Samy A F Morad, Myles C Cabot
Jun 27, 2013·Proceedings of the National Academy of Sciences of the United States of America·Tatyana O SharpeeJohn H Reynolds
Nov 10, 2013·Nature·Valerio ManteWilliam T Newsome
May 9, 2014·Proceedings of the National Academy of Sciences of the United States of America·Daniel L K YaminsJames J DiCarlo
Nov 7, 2014·PLoS Computational Biology·Seyed-Mahdi Khaligh-Razavi, Nikolaus Kriegeskorte
Dec 19, 2014·PLoS Computational Biology·Charles F CadieuJames J DiCarlo
Feb 27, 2015·Nature·Volodymyr MnihDemis Hassabis
May 9, 2015·Proceedings of the National Academy of Sciences of the United States of America·Arash AfrazJames J DiCarlo

Related Concepts

Body Regions
Sensory Cortex
Objective (Goal)
Neural Network Simulation
Structure of Cortex of Kidney
Cellular Component Organization
Sensory Process
Cerebral Cortex

Trending Feeds


Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

Chronic Fatigue Syndrome

Chronic fatigue syndrome is a disease characterized by unexplained disabling fatigue; the pathology of which is incompletely understood. Discover the latest research on chronic fatigue syndrome here.

Synapse Loss as Therapeutic Target in MS

As we age, the number of synapses present in the human brain starts to decline, but in neurodegenerative diseases this occurs at an accelerated rate. In MS, it has been shown that there is a reduction in synaptic density, which presents a potential target for treatment. Here is the latest research on synapse loss as a therapeutic target in MS.

Artificial Intelligence in Cardiac Imaging

Artificial intelligence (ai) techniques are increasingly applied to cardiovascular (cv) medicine in cardiac imaging analysis. Here is the latest research.

Position Effect Variegation

Position Effect Variagation occurs when a gene is inactivated due to its positioning near heterochromatic regions within a chromosome. Discover the latest research on Position Effect Variagation here.

Social Learning

Social learning involves learning new behaviors through observation, imitation and modeling. Follow this feed to stay up to date on the latest research.

Cell Atlas of the Human Eye

Constructing a cell atlas of the human eye will require transcriptomic and histologic analysis over the lifespan. This understanding will aid in the study of development and disease. Find the latest research pertaining to the Cell Atlas of the Human Eye here.

Single Cell Chromatin Profiling

Techniques like ATAC-seq and CUT&Tag have the potential to allow single cell profiling of chromatin accessibility, histones, and TFs. This will provide novel insight into cellular heterogeneity and cell states. Discover the latest research on single cell chromatin profiling here.

Genetic Screens in iPSC-derived Brain Cells

Genetic screening is a critical tool that can be employed to define and understand gene function and interaction. This feed focuses on genetic screens conducted using induced pluripotent stem cell (iPSC)-derived brain cells.

Related Papers

Shinrigaku kenkyu : The Japanese journal of psychology
Takashi TsuzukiTakashi Kusumi
Science China. Life Sciences
Si Wu, PeiJi Liang
Computational and Mathematical Methods in Medicine
Jinde CaoAhmed Elaiw
Nature Reviews. Neuroscience
Claudia WiedemannMonica Hoyos Flight
© 2020 Meta ULC. All rights reserved