Dopaminergic Control of the Exploration-Exploitation Trade-Off via the Basal Ganglia.

Frontiers in Neuroscience
Mark D HumphriesK N Gurney

Abstract

We continuously face the dilemma of choosing between actions that gather new information or actions that exploit existing knowledge. This "exploration-exploitation" trade-off depends on the environment: stability favors exploiting knowledge to maximize gains; volatility favors exploring new options and discovering new outcomes. Here we set out to reconcile recent evidence for dopamine's involvement in the exploration-exploitation trade-off with the existing evidence for basal ganglia control of action selection, by testing the hypothesis that tonic dopamine in the striatum, the basal ganglia's input nucleus, sets the current exploration-exploitation trade-off. We first advance the idea of interpreting the basal ganglia output as a probability distribution function for action selection. Using computational models of the full basal ganglia circuit, we showed that, under this interpretation, the actions of dopamine within the striatum change the basal ganglia's output to favor the level of exploration or exploitation encoded in the probability distribution. We also found that our models predict striatal dopamine controls the exploration-exploitation trade-off if we instead read-out the probability distribution from the target nucl...Continue Reading

Citations

Aug 1, 2012·Cognitive Neurodynamics·Simon M Vogt, Ulrich G Hofmann
Oct 3, 2013·Biological cybernetics·Romain D Cazé, Matthijs A A van der Meer
Mar 26, 2014·Cognitive, Affective & Behavioral Neuroscience·A Ross OttoBradley C Love
Jan 23, 2016·Trends in Cognitive Sciences·Arjen StolkIvan Toni
Nov 26, 2015·Nature Neuroscience·Arif A HamidJoshua D Berke
Nov 20, 2015·Frontiers in Computational Neuroscience·Thomas H B FitzGeraldKarl Friston
Jun 24, 2014·Journal of Physiology, Paris·Florian LesaintMehdi Khamassi
Feb 18, 2015·Cognitive Neuroscience·Karl FristonGiovanni Pezzulo
Aug 19, 2015·Psychiatry Research·Bariş O Yildirim, Jan J L Derksen
Dec 17, 2014·Neuropharmacology·Jérémie NaudéPhilippe Faure
Jun 16, 2015·Frontiers in Neuroscience·Alekhya MandaliAhmed A Moustafa
Aug 27, 2013·Journal of Physiology, Paris·Ignasi CosBenoît Girard
Nov 13, 2014·Frontiers in Behavioral Neuroscience·Shinae KwakMin W Jung
Mar 29, 2014·Frontiers in Systems Neuroscience·Aude Retailleau, Thomas Boraud
Jul 5, 2016·Neuroscience and Biobehavioral Reviews·Karl FristonGiovanni Pezzulo
Aug 11, 2016·Wiley Interdisciplinary Reviews. Cognitive Science·Roshan Cools
Sep 15, 2016·Journal of Cognitive Neuroscience·Ima TremplerRicarda I Schubotz
Oct 14, 2016·PLoS Computational Biology·Ayaka Kato, Kenji Morita
Nov 22, 2016·Neural Computation·Karl FristonGiovanni Pezzulo
Mar 1, 2019·PLoS Computational Biology·Moritz Möller, Rafal Bogacz
Oct 2, 2019·Nature Reviews. Neuroscience·Samuel J Gershman, Naoshige Uchida
Apr 18, 2020·The European Journal of Neuroscience·Jonathan E RubinTimothy Verstynen
Dec 7, 2017·Scientific Reports·Irene Cogliati DezzaWilliam Alexander
Jun 3, 2017·Cognitive, Affective & Behavioral Neuroscience·Asako ToyamaHideki Ohira
Jul 6, 2019·Psychopharmacology·Trevor W Robbins, Rudolf N Cardinal
Feb 5, 2021·NPJ Schizophrenia·Flurin CathomasStefan Kaiser
Oct 23, 2018·Neuropsychologia·Samuel J Gershman, Bastian Greshake Tzovaras
Mar 19, 2021·Frontiers in Behavioral Neuroscience·Robert WordenVictorita Neacsu
Mar 25, 2021·PLoS Computational Biology·John G MikhaelSamuel J Gershman
May 21, 2021·Neuroscience·Tom Gilbertson, Douglas Steele
Jun 15, 2021·Frontiers in Neural Circuits·Nihaad Paraouty, Todd M Mowery
Jun 23, 2021·Neural Networks : the Official Journal of the International Neural Network Society·Hiroyuki OhtaTatsuji Takahashi

❮ Previous
Next ❯

Related Concepts

Related Feeds

Basal Ganglia

Basal Ganglia are a group of subcortical nuclei in the brain associated with control of voluntary motor movements, procedural and habit learning, emotion, and cognition. Here is the latest research.

© 2022 Meta ULC. All rights reserved