Metaplasticity as a Neural Substrate for Adaptive Learning and Choice under Uncertainty

Neuron
Shiva FarashahiAlireza Soltani

Abstract

Value-based decision making often involves integration of reward outcomes over time, but this becomes considerably more challenging if reward assignments on alternative options are probabilistic and non-stationary. Despite the existence of various models for optimally integrating reward under uncertainty, the underlying neural mechanisms are still unknown. Here we propose that reward-dependent metaplasticity (RDMP) can provide a plausible mechanism for both integration of reward under uncertainty and estimation of uncertainty itself. We show that a model based on RDMP can robustly perform the probabilistic reversal learning task via dynamic adjustment of learning based on reward feedback, while changes in its activity signal unexpected uncertainty. The model predicts time-dependent and choice-specific learning rates that strongly depend on reward history. Key predictions from this model were confirmed with behavioral data from non-human primates. Overall, our results suggest that metaplasticity can provide a neural substrate for adaptive learning and choice under uncertainty.

References

Apr 1, 1996·Trends in Neurosciences·W C Abraham, M F Bear
Oct 10, 2002·Neural Networks : the Official Journal of the International Neural Network Society·Kenji Doya
Oct 10, 2002·Neural Networks : the Official Journal of the International Neural Network Society·John N J Reynolds, Jeffery R Wickens
Oct 10, 2002·Neural Networks : the Official Journal of the International Neural Network Society·Nathaniel D DawPeter Dayan
Oct 18, 2002·Neuron·Wolfram Schultz
Feb 11, 2003·Neural Networks : the Official Journal of the International Neural Network Society·Nicolas Schweighofer, Kenji Doya
Feb 22, 2005·Neuron·Stefano FusiL F Abbott
Jun 10, 2005·Neuron·Angela J Yu, Peter Dayan
Jul 19, 2005·Annual Review of Neuroscience·Gary Aston-Jones, Jonathan D Cohen
Apr 7, 2006·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Alireza Soltani, Xiao-Jing Wang
Jun 17, 2006·Nature·Nathaniel D DawRaymond J Dolan
Jun 20, 2006·Nature Neuroscience·Steven W KennerleyMatthew F S Rushworth
Jun 24, 2006·Trends in Cognitive Sciences·Aaron C CourvilleDavid S Touretzky
Aug 11, 2006·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Alan N HamptonJohn P O'Doherty
Oct 4, 2006·Neural Networks : the Official Journal of the International Neural Network Society·Alireza SoltaniXiao-Jing Wang
Mar 9, 2007·Annals of the New York Academy of Sciences·Kerstin Preuschoff, Peter Bossaerts
Jun 30, 2007·Annual Review of Neuroscience·Wolfram Schultz
Aug 7, 2007·Nature Neuroscience·Timothy E J BehrensMatthew F S Rushworth
Oct 5, 2007·Proceedings of the National Academy of Sciences of the United States of America·Michael J FrankKent E Hutchison
Mar 15, 2008·Science·Gianluigi MongilloMisha Tsodyks
Mar 28, 2008·Nature Neuroscience·Matthew F S Rushworth, Timothy E J Behrens
Apr 11, 2008·Nature Reviews. Neuroscience·Wickliffe C Abraham
Jan 13, 2009·Nature Neuroscience·Khaled MoussawiPeter W Kalivas
Oct 14, 2009·Proceedings of the National Academy of Sciences of the United States of America·Lea K KrugelHauke R Heekeren
Dec 17, 2009·Nature Neuroscience·Alireza Soltani, Xiao-Jing Wang
Sep 17, 2010·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Matthew R NassarJoshua I Gold
Oct 29, 2010·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Rafal RygulaAngela C Roberts
Feb 2, 2011·PLoS Computational Biology·Elise Payzan-LeNestour, Peter Bossaerts
Feb 15, 2011·Nature Neuroscience·Alberto BernacchiaXiao-Jing Wang
Jun 2, 2011·Frontiers in Human Neuroscience·Christoph MathysKlaas E Stephan
Oct 14, 2011·Frontiers in Neuroscience·Kerstin PreuschoffWolfgang Einhäuser
Jan 13, 2012·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Yael NivJohn P O'Doherty
Jun 5, 2012·Nature Neuroscience·Matthew R NassarJoshua I Gold
Jun 16, 2012·Frontiers in Neuroscience·Amy R Bland, Alexandre Schaefer
Apr 23, 2013·Trends in Neurosciences·Sarah R HulmeWickliffe C Abraham
Jun 25, 2013·Nature Neuroscience·Peter H RudebeckElisabeth A Murray
Aug 13, 2013·PLoS Computational Biology·Robert C WilsonJoshua I Gold
Jan 28, 2014·Neuron·Robert C WilsonYael Niv
Feb 5, 2014·Psychological Review·C R GallistelPeter E Latham
Mar 13, 2014·The Neuroscientist : a Review Journal Bringing Neurobiology, Neurology and Psychiatry·Florian Müller-Dahlhaus, Ulf Ziemann
Jan 13, 2015·Nature Neuroscience·Christopher H Donahue, Daeyeol Lee
Feb 13, 2015·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Vincent D CostaBruno B Averbeck
Jul 17, 2015·Journal of Neurophysiology·Kelly M J Diederen, Wolfram Schultz
Aug 21, 2015·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Anthony I JangBruno B Averbeck
Dec 1, 2015·Frontiers in Computational Neuroscience·Pierre Yger, Matthieu Gilson

Citations

Jun 29, 2017·PLoS Computational Biology·Peyman Khorsand, Alireza Soltani
Apr 4, 2018·Experimental Brain Research·Howard C CromwellWolfram Schultz
Nov 25, 2017·Nature Communications·Shiva FarashahiAlireza Soltani
Aug 12, 2018·Nature Communications·Jan ZimmermannKenway Louie
Jan 8, 2019·Cerebral Cortex·Michael J SiniscalchiAlex C Kwan
May 20, 2018·Scientific Reports·Zachary P Kilpatrick
May 31, 2019·Nature Reviews. Neuroscience·Alireza Soltani, A Izquierdo
Sep 11, 2019·Nature Human Behaviour·Shiva FarashahiAlireza Soltani
Dec 19, 2019·ELife·Jennifer L CookR Cools
Dec 19, 2019·Journal of Cognitive Neuroscience·Mohsen RakhshanAlireza Soltani
Jul 2, 2020·PLoS Computational Biology·Payam Piray, Nathaniel D Daw
Jul 30, 2020·Nature Communications·Marco K WittmannMatthew F S Rushworth
Aug 8, 2018·Scientific Reports·Thomas W ElstonDavid K Bilkey
Aug 26, 2020·Proceedings of the National Academy of Sciences of the United States of America·Mehran SpitmaanAlireza Soltani
Apr 10, 2019·PLoS Computational Biology·Micha Heilbron, Florent Meyniel
Nov 10, 2020·Journal of Cognitive Neuroscience·Alireza SoltaniTirin Moore
Nov 18, 2020·ELife·Bastien Blain, Robb B Rutledge
Jan 2, 2021·Trends in Cognitive Sciences·Ifat Levy, Daniela Schiller
Feb 3, 2021·PLoS Computational Biology·Sam GijsenFelix Blankenburg
Feb 24, 2018·Neurobiology of Learning and Memory·J Jane ZhangKarim Nader
May 24, 2021·Current Opinion in Behavioral Sciences·Alireza SoltaniDaeyeol Lee

Related Concepts

Related Feeds

Brain developing: Influences & Outcomes

This feed focuses on influences that affect the developing brain including genetics, fetal development, prenatal care, and gene-environment interactions. Here is the latest research in this field.

Related Papers

Neural Networks : the Official Journal of the International Neural Network Society
Alireza SoltaniXiao-Jing Wang
Journal of Behavior Therapy and Experimental Psychiatry
Yoshihiko KunisatoShigeto Yamawaki
© 2021 Meta ULC. All rights reserved