Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal

Psychological Review
Michael J Frank, Eric D Claus

Abstract

The authors explore the division of labor between the basal ganglia-dopamine (BG-DA) system and the orbitofrontal cortex (OFC) in decision making. They show that a primitive neural network model of the BG-DA system slowly learns to make decisions on the basis of the relative probability of rewards but is not as sensitive to (a) recency or (b) the value of specific rewards. An augmented model that explores BG-OFC interactions is more successful at estimating the true expected value of decisions and is faster at switching behavior when reinforcement contingencies change. In the augmented model, OFC areas exert top-down control on the BG and premotor areas by representing reinforcement magnitudes in working memory. The model successfully captures patterns of behavior resulting from OFC damage in decision making, reversal learning, and devaluation paradigms and makes additional predictions for the underlying source of these deficits.

References

Sep 1, 1992·Brain and Cognition·R J Davidson
Jul 1, 1990·Trends in Neurosciences·G E Alexander, M D Crutcher
Aug 1, 1972·Experimental Neurology·B Jones, M Mishkin
Jan 1, 1970·Brain : a Journal of Neurology·J M Kemp, T P Powell
Jan 1, 1983·Annual Review of Neuroscience·I CreeseS E Leff
Jan 1, 1982·Journal of Mathematical Biology·E Oja
Mar 30, 1981·Brain Research·C E RosenkildeJ M Fuster
May 1, 1981·Annals of Neurology·R G Robinson, B Szetela
Jan 30, 1981·Science·A Tversky, D Kahneman
Oct 1, 1994·Scientific American·A R Damasio
Dec 15, 1995·Annals of the New York Academy of Sciences·P S Goldman-Rakic
Sep 6, 1996·Science·B J KnowltonL R Squire
Dec 25, 1995·The Journal of Comparative Neurology·S T Carmichael, J L Price
Oct 29, 1996·Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences·E T Rolls
Jan 1, 1996·Critical Reviews in Neurobiology·S P WiseC R Gerfen
Mar 14, 1997·Science·W SchultzP R Montague
Oct 23, 1998·Archives of General Psychiatry·C M MazzantiD Goldman
Apr 9, 1999·Nature Neuroscience·J R Hollerman, W Schultz
May 5, 1999·Nature·L Tremblay, W Schultz
May 11, 1999·Trends in Cognitive Sciences·R J Davidson, W Irwin
Nov 27, 1999·Science·M S JogA M Graybiel
Mar 24, 2000·Cerebral Cortex·C CavadaF Reinoso-Suárez
May 23, 2000·Journal of Neural Transmission·U MüllerD Y von Cramon
Jun 9, 2000·Annual Review of Neuroscience·J E LeDoux
Oct 29, 2000·Trends in Neurosciences·S P OnnA A Grace
Oct 29, 2000·Trends in Neurosciences·C R Gerfen
Jan 3, 2001·Proceedings of the National Academy of Sciences of the United States of America·J K SeamansT J Sejnowski
Jan 3, 2001·Nature Neuroscience·J O'DohertyC Andrews
Apr 3, 2001·Annual Review of Neuroscience·E K Miller, J D Cohen
Apr 4, 2001·The European Journal of Neuroscience·D CentonzeP Calabresi

❮ Previous
Next ❯

Citations

Oct 7, 2008·Applied Psychophysiology and Biofeedback·Rex CannonDebora Baldwin
Jul 2, 2008·The Cerebellum·Patrizia ThomaIrene Daum
Sep 29, 2011·Nature Neuroscience·James F CavanaghMichael J Frank
Jun 25, 2013·Nature Neuroscience·Peter H RudebeckElisabeth A Murray
Dec 27, 2006·Nature Neuroscience·Alexander LerchnerBarry Richmond
Sep 25, 2009·Neuropsychopharmacology : Official Publication of the American College of Neuropsychopharmacology·Bernard W Balleine, John P O'Doherty
Oct 1, 2010·Neuropsychopharmacology : Official Publication of the American College of Neuropsychopharmacology·Y-Lan Boureau, Peter Dayan
Jan 7, 2011·Neuropsychopharmacology : Official Publication of the American College of Neuropsychopharmacology·Catharine A WinstanleyRobert D Rogers
Jul 16, 2010·Neuropsychopharmacology : Official Publication of the American College of Neuropsychopharmacology·Michael J Frank, John A Fossella
Dec 14, 2006·Neuropsychopharmacology : Official Publication of the American College of Neuropsychopharmacology·Michael J FrankErik Willcutt
Feb 15, 2007·Neuropsychopharmacology : Official Publication of the American College of Neuropsychopharmacology·Buyean LeeJames David Jentsch
Dec 9, 2010·Journal of Clinical and Experimental Neuropsychology·Michele PolettiUbaldo Bonuccelli
May 12, 2009·Cerebral Cortex·Nicholas D Walsh, Mary L Phillips
Nov 17, 2009·Cerebral Cortex·Dara G GhahremaniRussell A Poldrack
Aug 10, 2013·Social Cognitive and Affective Neuroscience·Simone KühnJürgen Gallinat
Jul 2, 2008·Schizophrenia Bulletin·James M GoldErin A Heerey
Sep 16, 2008·Schizophrenia Bulletin·Michael J Frank
Nov 16, 2013·Journal of Neurologic Physical Therapy : JNPT·Keith LohseH F Machiel Van der Loos
Apr 13, 2007·Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences·Thomas E HazyRandall C O'reilly
Apr 13, 2007·Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences·Michael J FrankScott J Sherman
Jul 19, 2011·European Journal of Neurology : the Official Journal of the European Federation of Neurological Societies·P L PanH F Shang
Aug 30, 2008·Journal of Cognitive Neuroscience·Thorsten KahntJana Wrase
Oct 1, 2008·Journal of Cognitive Neuroscience·Daphna ShohamyMark A Gluck
Nov 21, 2008·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Ahmed A MoustafaMichael J Frank
Mar 18, 2011·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Hannah F ClarkeAngela C Roberts
Mar 27, 2009·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Gerhard JochamMarkus Ullsperger
Mar 13, 2009·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Eyal Yaacov Kimchi, Mark Laubach

❮ Previous
Next ❯

Related Concepts

Related Feeds

Basal Ganglia

Basal Ganglia are a group of subcortical nuclei in the brain associated with control of voluntary motor movements, procedural and habit learning, emotion, and cognition. Here is the latest research.