Prefrontal cortex as a meta-reinforcement learning system

Nature Neuroscience
Jane X WangMatthew M Botvinick

Abstract

Over the past 20 years, neuroscience research on reward-based learning has converged on a canonical model, under which the neurotransmitter dopamine 'stamps in' associations between situations, actions and rewards by modulating the strength of synaptic connections between neurons. However, a growing number of recent findings have placed this standard model under strain. We now draw on recent advances in artificial intelligence to introduce a new theory of reward-based learning. Here, the dopamine system trains another part of the brain, the prefrontal cortex, to operate as its own free-standing learning system. This new perspective accommodates the findings that motivated the standard model, but also deals gracefully with a wider range of observations, providing a fresh foundation for future research.

References

Oct 23, 1997·Neural Computation·S Hochreiter, J Schmidhuber
Jul 31, 2001·Trends in Neurosciences·X J Wang
Oct 10, 2002·Neural Networks : the Official Journal of the International Neural Network Society·Shin IshiiJunichiro Yoshimoto
Feb 11, 2003·Neural Networks : the Official Journal of the International Neural Network Society·Nicolas Schweighofer, Kenji Doya
Mar 9, 2004·Nature Neuroscience·Dominic J BarracloughDaeyeol Lee
Nov 6, 2004·Science·Michael J FrankRandall C O'reilly
May 11, 2005·Proceedings of the National Academy of Sciences of the United States of America·Nicolas P RougierRandall C O'Reilly
Apr 7, 2006·Journal of the Experimental Analysis of Behavior·Brian Lau, Paul W Glimcher
Apr 25, 2006·Nature·Camillo Padoa-Schioppa, John A Assad
Aug 11, 2006·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Alan N HamptonJohn P O'Doherty
Aug 7, 2007·Nature Neuroscience·Timothy E J BehrensMatthew F S Rushworth
Jan 1, 1949·Psychological Review·H F HARLOW
Mar 28, 2008·Nature Neuroscience·Matthew F S Rushworth, Timothy E J Behrens
Oct 3, 2008·Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences·Hyojung Seo, Daeyeol Lee
Jun 12, 2010·Journal of Neurophysiology·Ethan S Bromberg-MartinOkihide Hikosaka
Dec 15, 2010·PLoS Computational Biology·Daniel E Acuña, Paul Schrater
Mar 26, 2011·Neuron·Nathaniel D DawRaymond J Dolan
Nov 1, 2011·Nature Neuroscience·Yuji K TakahashiGeoffrey Schoenbaum
Apr 25, 2012·Psychonomic Bulletin & Review·James R Schmidt, Jan De Houwer
Jun 12, 2012·Neuron·Moonsang SeoBruno B Averbeck
Oct 17, 2012·Neuroscience Research·Hiroyuki Nakahara, Okihide Hikosaka
Jan 4, 2013·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Hoseok KimMin Whan Jung
Jan 16, 2013·Progress in Brain Research·Mehdi KhamassiEmmanuel Procyk
Nov 10, 2013·Nature·Valerio ManteWilliam T Newsome
Dec 8, 2015·Nature Neuroscience·Chun Yun ChangGeoffrey Schoenbaum
Jan 1, 2016·Current Opinion in Behavioral Sciences·Christopher H Chatham, David Badre
Jul 29, 2016·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Stephanie C Y ChanKenneth A Norman
Sep 14, 2016·Annual Review of Psychology·Samuel J Gershman, Nathaniel D Daw
Sep 14, 2016·Nature Communications·Ken-Ichiro TsutsuiWolfram Schultz
Sep 24, 2016·Neuron·Nicolas W SchuckYael Niv
Aug 2, 2017·Nature Neuroscience·Kevin J MillerCarlos D Brody

❮ Previous
Next ❯

Citations

Jun 12, 2019·PLoS Computational Biology·Amir DezfouliBernard W Balleine
Dec 19, 2019·ELife·Jennifer L CookRoshan Cools
Apr 25, 2020·PLoS Computational Biology·Ramon BartoloBruno B Averbeck
Apr 28, 2020·The Neuroscientist : a Review Journal Bringing Neurobiology, Neurology and Psychiatry·Kelly M J Diederen, Paul C Fletcher
Aug 12, 2020·ELife·Andreea Oliviana DiaconescuPhilippe N Tobler
Sep 7, 2018·Behavioural Pharmacology·Bolton K H ChauTrevor T-J Chong
Jan 24, 2019·Proceedings of the National Academy of Sciences of the United States of America·M Mitchell Waldrop
Oct 30, 2019·Nature Neuroscience·Blake A RichardsKonrad P Kording
Aug 4, 2020·Journal of Alzheimer's Disease : JAD·Daniele CaligioreGianluca Baldassarre
Sep 3, 2020·Nature Reviews. Neuroscience·Anne G E Collins, Jeffrey Cockburn
Oct 17, 2018·PloS One·Silvan Quax, Marcel van Gerven
Jun 12, 2019·Nature Neuroscience·Nicolas Y MasseDavid J Freedman
Jun 9, 2020·Frontiers in Computational Neuroscience·Ryan SmithKarl J Friston
Sep 27, 2018·Journal of Child Psychology and Psychiatry, and Allied Disciplines·Tobias U HauserRaymond J Dolan
Oct 23, 2019·Proceedings of the National Academy of Sciences of the United States of America·Robert KimTerrence J Sejnowski
Mar 19, 2020·Nature Human Behaviour·Adam Bulley, Daniel L Schacter
Sep 24, 2020·Nature Communications·Shirley MarkTimothy E J Behrens
Jan 17, 2020·Nature·Will DabneyMatthew Botvinick
Sep 11, 2020·Physical Review Letters·Christian KlosRaoul-Martin Memmesheimer
Sep 2, 2020·Nature Communications·Aurelio CorteseMitsuo Kawato
Jan 12, 2019·Frontiers in Neuroinformatics·Hananel HazanRobert Kozma
Nov 25, 2020·Proceedings of the National Academy of Sciences of the United States of America·Maria K Eckstein, Anne G E Collins
Jan 6, 2021·Cognitive Science·Thomas PouncySamuel J Gershman
Jan 29, 2021·Current Opinion in Neurobiology·Dhruva V Raman, Timothy O'Leary
Jan 30, 2021·Nature Human Behaviour·Momchil S TomovSamuel J Gershman
Feb 17, 2021·Brain Informatics·Frederic Alexandre
Feb 18, 2021·Trends in Cognitive Sciences·Clay B Holroyd, Tom Verguts
Dec 1, 2020·Frontiers in Cellular Neuroscience·Charlotte PietteLaurent Venance
Nov 7, 2020·Proceedings of the National Academy of Sciences of the United States of America·Ben TsudaTerrence J Sejnowski
Apr 17, 2020·Trends in Cognitive Sciences·Kate ErgoTom Verguts
Mar 6, 2021·Frontiers in Computational Neuroscience·Hongjian FangFeifei Zhao
Sep 11, 2019·Neural Networks : the Official Journal of the International Neural Network Society·Devdhar PatelRobert Kozma
Nov 2, 2019·Current Opinion in Neurobiology·Simon MusallAnne K Churchland
Feb 3, 2020·Trends in Cognitive Sciences·Nicolas Y MasseDavid J Freedman
Mar 16, 2021·Frontiers in Neuroscience·Carlos SilvaKristin L Hillman
Jun 4, 2020·Current Opinion in Behavioral Sciences·Guangyu Robert YangKanaka Rajan
Mar 24, 2021·Trends in Cognitive Sciences·Eva Zita Patai, Hugo J Spiers
Oct 4, 2019·Neural Networks : the Official Journal of the International Neural Network Society·Alexandre PittiSofiane Boucenna

❮ Previous
Next ❯

Software Mentioned

RL
meta
Advantage Actor - Critic
Psychlab
SARSA

Related Concepts

Trending Feeds

COVID-19

Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

Blastomycosis

Blastomycosis fungal infections spread through inhaling Blastomyces dermatitidis spores. Discover the latest research on blastomycosis fungal infections here.

Nuclear Pore Complex in ALS/FTD

Alterations in nucleocytoplasmic transport, controlled by the nuclear pore complex, may be involved in the pathomechanism underlying multiple neurodegenerative diseases including Amyotrophic Lateral Sclerosis and Frontotemporal Dementia. Here is the latest research on the nuclear pore complex in ALS and FTD.

Applications of Molecular Barcoding

The concept of molecular barcoding is that each original DNA or RNA molecule is attached to a unique sequence barcode. Sequence reads having different barcodes represent different original molecules, while sequence reads having the same barcode are results of PCR duplication from one original molecule. Discover the latest research on molecular barcoding here.

Chronic Fatigue Syndrome

Chronic fatigue syndrome is a disease characterized by unexplained disabling fatigue; the pathology of which is incompletely understood. Discover the latest research on chronic fatigue syndrome here.

Evolution of Pluripotency

Pluripotency refers to the ability of a cell to develop into three primary germ cell layers of the embryo. This feed focuses on the mechanisms that underlie the evolution of pluripotency. Here is the latest research.

Position Effect Variegation

Position Effect Variagation occurs when a gene is inactivated due to its positioning near heterochromatic regions within a chromosome. Discover the latest research on Position Effect Variagation here.

STING Receptor Agonists

Stimulator of IFN genes (STING) are a group of transmembrane proteins that are involved in the induction of type I interferon that is important in the innate immune response. The stimulation of STING has been an active area of research in the treatment of cancer and infectious diseases. Here is the latest research on STING receptor agonists.

Microbicide

Microbicides are products that can be applied to vaginal or rectal mucosal surfaces with the goal of preventing, or at least significantly reducing, the transmission of sexually transmitted infections. Here is the latest research on microbicides.

Related Papers

Science
Barry J RichmondMunetaka Shidara
Zhonghua yi xue za zhi
Li-wei ZhangPeng Zhang
The Journal of Neuroscience : the Official Journal of the Society for Neuroscience
Kei OyamaKen-Ichiro Tsutsui
Journal of the American Medical Directors Association
Esther R Gold
© 2022 Meta ULC. All rights reserved