Actions, action sequences and habits: evidence that goal-directed and habitual action control are hierarchically organized

PLoS Computational Biology
Amir Dezfouli, Bernard W Balleine

Abstract

Behavioral evidence suggests that instrumental conditioning is governed by two forms of action control: a goal-directed and a habit learning process. Model-based reinforcement learning (RL) has been argued to underlie the goal-directed process; however, the way in which it interacts with habits and the structure of the habitual process has remained unclear. According to a flat architecture, the habitual process corresponds to model-free RL, and its interaction with the goal-directed process is coordinated by an external arbitration mechanism. Alternatively, the interaction between these systems has recently been argued to be hierarchical, such that the formation of action sequences underlies habit learning and a goal-directed process selects between goal-directed actions and habitual sequences of actions to reach the goal. Here we used a two-stage decision-making task to test predictions from these accounts. The hierarchical account predicts that, because they are tied to each other as an action sequence, selecting a habitual action in the first stage will be followed by a habitual action in the second stage, whereas the flat account predicts that the statuses of the first and second stage actions are independent of each other....Continue Reading

References

Jan 1, 1976·Archives of Sexual Behavior·J Erwin, T Maple
May 1, 1966·Journal of Experimental Psychology·R W Pew
Mar 14, 1997·Science·W SchultzP R Montague
Jun 9, 2000·Annual Review of Neuroscience·W Schultz, A Dickinson
Mar 29, 2003·Neural Networks : the Official Journal of the International Neural Network Society·K Doya
Apr 7, 2006·Journal of the Experimental Analysis of Behavior·Brian Lau, Paul W Glimcher
Oct 4, 2006·Psychological Review·Richard P Cooper, Tim Shallice
Nov 8, 2006·Journal of Experimental Psychology. General·Darryl W Schneider, Gordon D Logan
Jun 12, 2008·Nature Reviews. Neuroscience·Antonio RangelP Read Montague
Jun 19, 2008·Annual Review of Neuroscience·Ann M Graybiel
Jul 30, 2008·The Behavioral and Brain Sciences·A David RedishAdam Johnson
Sep 19, 2008·Trends in Cognitive Sciences·Frederick Verbruggen, Gordon D Logan
Mar 25, 2009·NeuroImage·Klaas Enno StephanKarl J Friston
Jun 26, 2009·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Sean B OstlundBernard W Balleine
Jul 14, 2009·Cognition·Jeremy R Reynolds, Randall C O'Reilly
Aug 7, 2009·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Makoto Ito, Kenji Doya
Sep 25, 2009·Neuropsychopharmacology : Official Publication of the American College of Neuropsychopharmacology·Bernard W Balleine, John P O'Doherty
Nov 27, 2009·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Hoseok KimMin Whan Jung
Mar 20, 2010·PLoS Computational Biology·Will D PennyAlex P Leff
Jun 1, 2000·Cognitive Neuropsychology·R Cooper, T Shallice
Dec 25, 2010·Journal of Motor Behavior·Alan FerminKenji Doya
Mar 26, 2011·Neuron·Nathaniel D DawRaymond J Dolan
Jun 4, 2011·PLoS Computational Biology·Mehdi KeramatiPayam Piray
Jul 28, 2011·Neuron·José J F Ribas-FernandesMatthew M Botvinick
Jan 10, 2012·Trends in Cognitive Sciences·Clay B Holroyd, Nick Yeung
Apr 11, 2012·The European Journal of Neuroscience·Amir Dezfouli, Bernard W Balleine
Aug 14, 2012·Neuron·Klaus WunderlichRaymond J Dolan
Mar 29, 2013·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Carlos DiukYael Niv

❮ Previous
Next ❯

Citations

Mar 22, 2014·Cognitive, Affective & Behavioral Neuroscience·Peter Dayan, Kent C Berridge
May 7, 2014·Cognitive, Affective & Behavioral Neuroscience·S de WitK R Ridderinkhof
Nov 30, 2013·Science·Leopold KongMansun Law
Jul 9, 2014·Proceedings of the National Academy of Sciences of the United States of America·Lu DengPei Zhang
Oct 1, 2014·Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences·Matthew Botvinick, Ari Weinstein
Oct 1, 2014·Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences·Amir DezfouliBernard W Balleine
Oct 16, 2015·Proceedings of the National Academy of Sciences of the United States of America·Fiery Cushman, Adam Morris
Oct 23, 2015·Proceedings of the National Academy of Sciences of the United States of America·Nathaniel D Daw
Sep 12, 2015·Annual Review of Psychology·Wendy Wood, Dennis Rünger
Mar 22, 2014·Topics in Cognitive Science·Peter Dayan
Jul 2, 2015·Behaviour Research and Therapy·Tom HildebrandtDaniela Schiller
Mar 25, 2015·Cognitive, Affective & Behavioral Neuroscience·Claire M GillanNathaniel D Daw
Feb 13, 2015·Proceedings of the National Academy of Sciences of the United States of America·Quentin J M HuysJonathan P Roiser
Dec 3, 2014·Frontiers in Human Neuroscience·Gloria Balderas
Nov 19, 2014·Frontiers in Human Neuroscience·Javier Bernacer, Jose Ignacio Murillo
Aug 15, 2014·PLoS Computational Biology·Kanghoon JungJaeseung Jeong
Aug 27, 2016·PLoS Computational Biology·Wouter KoolSamuel J Gershman
Nov 30, 2016·The Quarterly Journal of Experimental Psychology : QJEP·Omar D PérezAnthony Dickinson
Sep 14, 2016·Annual Review of Psychology·Samuel J Gershman, Nathaniel D Daw
Oct 1, 2016·Annual Review of Psychology·John P O'DohertyWolfgang M Pauli
Jul 28, 2016·Genes, Brain, and Behavior·C M Gremel, D M Lovinger
Dec 13, 2016·Cognitive Psychology·Oliver HerbortWilfried Kunde
Jul 22, 2017·Autism Research : Official Journal of the International Society for Autism Research·Krishna SubramanianGene J Blatt
Jan 21, 2018·Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences·Barry J EverittDavid Belin
Jun 8, 2018·CNS Neuroscience & Therapeutics·Tiziana Marilena FlorioMarcello Alecci
Dec 11, 2019·Journal of Neuroscience Research·Ellen P WoonShannon L Gourley
Apr 16, 2020·The Behavioral and Brain Sciences·Fiery Cushman
Apr 3, 2020·The European Journal of Neuroscience·Javier Baladron, Fred H Hamker
Apr 22, 2020·Nature Communications·Arkady Konovalov, Ian Krajbich
Jun 23, 2020·PLoS Computational Biology·Bruno MirandaSteven W Kennerley
Jul 4, 2020·Neuropsychopharmacology : Official Publication of the American College of Neuropsychopharmacology·Quentin J M HuysMichael J Frank
Sep 26, 2017·Frontiers in Computational Neuroscience·Cristóbal Moënne-LoccozDiego Cosmelli
Jul 8, 2020·Nature Human Behaviour·Carolina Feher da Silva, Todd A Hare
Jan 10, 2017·Frontiers in Behavioral Neuroscience·Zsuzsika SjoerdsAnnette Horstmann
Aug 21, 2018·Frontiers in Integrative Neuroscience·Brendan M Hasz, A David Redish

❮ Previous
Next ❯

Software Mentioned

DerApproximator
R package LanguageR
IPOPT
R
R package ‘ party ’
spm8
R package lme4

Related Concepts

Trending Feeds

COVID-19

Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

Blastomycosis

Blastomycosis fungal infections spread through inhaling Blastomyces dermatitidis spores. Discover the latest research on blastomycosis fungal infections here.

Nuclear Pore Complex in ALS/FTD

Alterations in nucleocytoplasmic transport, controlled by the nuclear pore complex, may be involved in the pathomechanism underlying multiple neurodegenerative diseases including Amyotrophic Lateral Sclerosis and Frontotemporal Dementia. Here is the latest research on the nuclear pore complex in ALS and FTD.

Applications of Molecular Barcoding

The concept of molecular barcoding is that each original DNA or RNA molecule is attached to a unique sequence barcode. Sequence reads having different barcodes represent different original molecules, while sequence reads having the same barcode are results of PCR duplication from one original molecule. Discover the latest research on molecular barcoding here.

Chronic Fatigue Syndrome

Chronic fatigue syndrome is a disease characterized by unexplained disabling fatigue; the pathology of which is incompletely understood. Discover the latest research on chronic fatigue syndrome here.

Evolution of Pluripotency

Pluripotency refers to the ability of a cell to develop into three primary germ cell layers of the embryo. This feed focuses on the mechanisms that underlie the evolution of pluripotency. Here is the latest research.

Position Effect Variegation

Position Effect Variagation occurs when a gene is inactivated due to its positioning near heterochromatic regions within a chromosome. Discover the latest research on Position Effect Variagation here.

STING Receptor Agonists

Stimulator of IFN genes (STING) are a group of transmembrane proteins that are involved in the induction of type I interferon that is important in the innate immune response. The stimulation of STING has been an active area of research in the treatment of cancer and infectious diseases. Here is the latest research on STING receptor agonists.

Microbicide

Microbicides are products that can be applied to vaginal or rectal mucosal surfaces with the goal of preventing, or at least significantly reducing, the transmission of sexually transmitted infections. Here is the latest research on microbicides.

Related Papers

The European Journal of Neuroscience
Amir Dezfouli, Bernard W Balleine
Drug Discovery Today. Disease Models
Sean B Ostlund, Bernard W Balleine
The Journal of Neuroscience : the Official Journal of the Society for Neuroscience
Lars Schwabe, Oliver T Wolf
© 2021 Meta ULC. All rights reserved