A spiking neural network model of model-free reinforcement learning with high-dimensional sensory input and perceptual ambiguity

PloS One
Takashi NakanoKenji Doya

Abstract

A theoretical framework of reinforcement learning plays an important role in understanding action selection in animals. Spiking neural networks provide a theoretically grounded means to test computational hypotheses on neurally plausible algorithms of reinforcement learning through numerical simulation. However, most of these models cannot handle observations which are noisy, or occurred in the past, even though these are inevitable and constraining features of learning in real environments. This class of problem is formally known as partially observable reinforcement learning (PORL) problems. It provides a generalization of reinforcement learning to partially observable domains. In addition, observations in the real world tend to be rich and high-dimensional. In this work, we use a spiking neural network model to approximate the free energy of a restricted Boltzmann machine and apply it to the solution of PORL problems with high-dimensional observations. Our spiking network model solves maze tasks with perceptually ambiguous high-dimensional observations without knowledge of the true environment. An extended model with working memory also solves history-dependent tasks. The way spiking neural networks handle PORL problems may ...Continue Reading

References

Mar 14, 1997·Science·W SchultzP R Montague
Sep 7, 2001·Nature·J N ReynoldsJ R Wickens
Aug 16, 2002·Neural Computation·Geoffrey E Hinton
Sep 10, 2002·Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences·Earl K MillerJonathan D Wallis
Oct 10, 2002·Neural Networks : the Official Journal of the International Neural Network Society·Kenji Doya
Nov 29, 2005·Science·Kazuyuki SamejimaMinoru Kimura
Aug 29, 2006·Nature·David J Freedman, John A Assad
Oct 23, 2008·Biological cybernetics·Patrick D RobertsGerardo Lafferriere
Jan 16, 2009·The Journal of Neuroscience : the Official Journal of the Society for Neuroscience·Wakoto MatsudaTakeshi Kaneko
Feb 7, 2009·Neural Computation·Wiebke PotjansMarkus Diesmann
Jul 21, 2009·Neural Networks : the Official Journal of the International Neural Network Society·Sohrab SaebJochen Triesch
Sep 3, 2010·PLoS Computational Biology·Botond Szatmáry, Eugene M Izhikevich
Mar 8, 2011·PLoS Computational Biology·Martin Boerlin, Sophie Denève

❮ Previous
Next ❯

Citations

Jun 2, 2020·Frontiers in Neuroscience·Ahana GangopadhyayShantanu Chakrabartty
Oct 22, 2019·Frontiers in Neurorobotics·Jacques KaiserRüdiger Dillmann

❮ Previous
Next ❯

Methods Mentioned

BETA
feature extraction

Software Mentioned

NEST
aFE
iFE
SARSA
FERL
PORL

Related Concepts

Trending Feeds

COVID-19

Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

Blastomycosis

Blastomycosis fungal infections spread through inhaling Blastomyces dermatitidis spores. Discover the latest research on blastomycosis fungal infections here.

Nuclear Pore Complex in ALS/FTD

Alterations in nucleocytoplasmic transport, controlled by the nuclear pore complex, may be involved in the pathomechanism underlying multiple neurodegenerative diseases including Amyotrophic Lateral Sclerosis and Frontotemporal Dementia. Here is the latest research on the nuclear pore complex in ALS and FTD.

Applications of Molecular Barcoding

The concept of molecular barcoding is that each original DNA or RNA molecule is attached to a unique sequence barcode. Sequence reads having different barcodes represent different original molecules, while sequence reads having the same barcode are results of PCR duplication from one original molecule. Discover the latest research on molecular barcoding here.

Chronic Fatigue Syndrome

Chronic fatigue syndrome is a disease characterized by unexplained disabling fatigue; the pathology of which is incompletely understood. Discover the latest research on chronic fatigue syndrome here.

Evolution of Pluripotency

Pluripotency refers to the ability of a cell to develop into three primary germ cell layers of the embryo. This feed focuses on the mechanisms that underlie the evolution of pluripotency. Here is the latest research.

Position Effect Variegation

Position Effect Variagation occurs when a gene is inactivated due to its positioning near heterochromatic regions within a chromosome. Discover the latest research on Position Effect Variagation here.

STING Receptor Agonists

Stimulator of IFN genes (STING) are a group of transmembrane proteins that are involved in the induction of type I interferon that is important in the innate immune response. The stimulation of STING has been an active area of research in the treatment of cancer and infectious diseases. Here is the latest research on STING receptor agonists.

Microbicide

Microbicides are products that can be applied to vaginal or rectal mucosal surfaces with the goal of preventing, or at least significantly reducing, the transmission of sexually transmitted infections. Here is the latest research on microbicides.

Related Papers

The Journal of Neuroscience : the Official Journal of the Society for Neuroscience
Jelena Jovancevic-Misic, Mary Hayhoe
Neural Networks : the Official Journal of the International Neural Network Society
Angela J Yu, Peter Dayan
© 2022 Meta ULC. All rights reserved