Model-based spatial navigation in the hippocampus-ventral striatum circuit: A computational analysis

PLoS Computational Biology
Ivilin Peev StoianovGiovanni Pezzulo

Abstract

While the neurobiology of simple and habitual choices is relatively well known, our current understanding of goal-directed choices and planning in the brain is still limited. Theoretical work suggests that goal-directed computations can be productively associated to model-based (reinforcement learning) computations, yet a detailed mapping between computational processes and neuronal circuits remains to be fully established. Here we report a computational analysis that aligns Bayesian nonparametrics and model-based reinforcement learning (MB-RL) to the functioning of the hippocampus (HC) and the ventral striatum (vStr)-a neuronal circuit that increasingly recognized to be an appropriate model system to understand goal-directed (spatial) decisions and planning mechanisms in the brain. We test the MB-RL agent in a contextual conditioning task that depends on intact hippocampus and ventral striatal (shell) function and show that it solves the task while showing key behavioral and neuronal signatures of the HC-vStr circuit. Our simulations also explore the benefits of biological forms of look-ahead prediction (forward sweeps) during both learning and control. This article thus contributes to fill the gap between our current understa...Continue Reading

References

Mar 22, 1982·Proceedings of the Royal Society of London. Series B, Containing Papers of a Biological Character·D Marr, L Vaina
Jul 29, 1994·Science·M A Wilson, B L McNaughton
Apr 1, 1996·Current Opinion in Neurobiology·R U MullerJ S Taube
Mar 14, 1997·Science·W SchultzP R Montague
Mar 12, 2005·Proceedings of the National Academy of Sciences of the United States of America·V HokB Poucet
Apr 8, 2006·Neuron·George Dragoi, György Buzsáki
Apr 21, 2006·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Mark C Fuhs, David S Touretzky
Jul 22, 2006·Nature Reviews. Neuroscience·Bruce L McNaughtonMay-Britt Moser
Nov 10, 2006·Hippocampus·Trygve SolstadGaute T Einevoll
Jan 20, 2007·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Vincent HokBruno Poucet
Apr 13, 2007·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Vivian V ValentinJohn P O'Doherty
Jun 29, 2007·Hippocampus·Neil BurgessJohn O'Keefe
Aug 1, 2007·Hippocampus·David J Foster, Matthew A Wilson
Sep 6, 2007·PLoS Computational Biology·Mathias FranziusLaurenz Wiskott
Sep 11, 2007·Nature Neuroscience·Kamran Diba, György Buzsáki
Nov 9, 2007·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Adam Johnson, A David Redish
Jan 22, 2008·Progress in Neurobiology·Tobias Kalenscher, Cyriel M A Pennartz
Apr 19, 2008·Trends in Cognitive Sciences·Matthew M Botvinick
Jul 4, 2008·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Rutsuko ItoBarry J Everitt
Jul 5, 2008·Science·Kirsten Brun KjelstrupMay-Britt Moser
Nov 8, 2008·PLoS Computational Biology·Karl Friston
Feb 20, 2009·Frontiers in Integrative Neuroscience·Matthijs A A van der Meer, A David Redish
Jun 17, 2009·Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences·John Lisman, A D Redish
Aug 19, 2009·PLoS Biology·Carien S LansinkCyriel M A Pennartz
Nov 3, 2010·Psychological Review·Adam N SanbornDaniel J Navarro
Jan 12, 2011·Frontiers in Neuroscience·Matthijs A A van der Meer, A David Redish
Feb 18, 2011·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Michael A McDannaldGeoffrey Schoenbaum
Mar 26, 2011·Neuron·Nathaniel D DawRaymond J Dolan
Jun 4, 2011·PLoS Computational Biology·Mehdi KeramatiPayam Piray
Jul 13, 2011·Frontiers in Neuroscience·Giovanni Pezzulo, Francesco Rigoli
Aug 27, 2011·Neuron·Lisa M GiocomoEdvard I Moser
Sep 6, 2011·Trends in Neurosciences·C M A PennartzT W Robbins
Apr 6, 2012·PLoS Biology·Anne Collins, Etienne Koechlin
Apr 12, 2012·The Neuroscientist : a Review Journal Bringing Neurobiology, Neurology and Psychiatry·Matthijs van der MeerA David Redish
Jul 19, 2012·Cognitive Processing·Fabian Chersi, Giovanni Pezzulo
Sep 8, 2012·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Carien S LansinkCyriel M A Pennartz
Jan 30, 2013·Psychological Review·Anne G E Collins, Michael J Frank
Feb 22, 2013·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Vincent HokBruno Poucet
Apr 19, 2013·Nature·Brad E Pfeiffer, David J Foster
Jul 31, 2013·Journal of Physiology, Paris·Uğur M Erdem, Michael E Hasselmo
Oct 22, 2013·Neuron·Ray J Dolan, Peter Dayan
Dec 19, 2013·PLoS Computational Biology·Will D PennyNeil Burgess
May 31, 2014·Science·Maël DonosoEtienne Koechlin

❮ Previous
Next ❯

Citations

Aug 28, 2020·Frontiers in Computational Neuroscience·Edgar Bermudez-ContrerasAaron Wilber
Apr 4, 2019·The Behavioral and Brain Sciences·Giovanni Pezzulo, Karl J Friston

❮ Previous
Next ❯

Software Mentioned

vStr
DYNA
MB
RL
swControl
HC

Related Concepts

Related Feeds

Basal Ganglia

Basal Ganglia are a group of subcortical nuclei in the brain associated with control of voluntary motor movements, procedural and habit learning, emotion, and cognition. Here is the latest research.