Kernel-based least squares policy iteration for reinforcement learning

IEEE Transactions on Neural Networks
Xin XuXicheng Lu

Abstract

In this paper, we present a kernel-based least squares policy iteration (KLSPI) algorithm for reinforcement learning (RL) in large or continuous state spaces, which can be used to realize adaptive feedback control of uncertain dynamic systems. By using KLSPI, near-optimal control policies can be obtained without much a priori knowledge on dynamic models of control plants. In KLSPI, Mercer kernels are used in the policy evaluation of a policy iteration process, where a new kernel-based least squares temporal-difference algorithm called KLSTD-Q is proposed for efficient policy evaluation. To keep the sparsity and improve the generalization ability of KLSTD-Q solutions, a kernel sparsification procedure based on approximate linear dependency (ALD) is performed. Compared to the previous works on approximate RL methods, KLSPI makes two progresses to eliminate the main difficulties of existing results. One is the better convergence and (near) optimality guarantee by using the KLSTD-Q algorithm for policy evaluation with high precision. The other is the automatic feature selection using the ALD-based kernel sparsification. Therefore, the KLSPI algorithm provides a general RL method with generalization performance and convergence guara...Continue Reading

References

Feb 6, 2008·IEEE Transactions on Neural Networks·J Moody, M Saffell
Jan 1, 1997·IEEE Transactions on Neural Networks·D V Prokhorov, D C Wunsch

Citations

Apr 30, 2014·Evidence-based Complementary and Alternative Medicine : ECAM·Chien-Shan ChengJianping Chen
Jan 13, 2015·IEEE Transactions on Neural Networks and Learning Systems·Yu JiangZhong-Ping Jiang
Dec 2, 2014·IEEE Transactions on Neural Networks and Learning Systems·Yan-Jun LiuDong-Juan Li
Apr 16, 2014·IEEE Transactions on Cybernetics·Kei SendaTakehisa Kohda
May 9, 2014·IEEE Transactions on Neural Networks and Learning Systems·Xingguo ChenRuili Wang
May 9, 2014·IEEE Transactions on Neural Networks and Learning Systems·Chunlin ChenTzyh-Jong Tarn
May 9, 2014·IEEE Transactions on Neural Networks and Learning Systems·Xin XuHaibo He
Mar 10, 2015·IEEE Transactions on Neural Networks and Learning Systems·Derong LiuDing Wang
May 9, 2014·IEEE Transactions on Neural Networks and Learning Systems·Matthieu Geist, Olivier Pietquin
May 29, 2007·IEEE Transactions on Neural Networks·Thomas HanselmannAnthony Zaknich
May 10, 2008·IEEE Transactions on Neural Networks·Sheng ChenLajos Hanzo
Jul 1, 2012·IEEE Transactions on Neural Networks and Learning Systems·Travis Dierks, Sarangapani Jagannathan
Sep 29, 2011·IEEE Transactions on Neural Networks·Yu Jiang, Zhong-Ping Jiang
Oct 13, 2011·IEEE Transactions on Neural Networks·Xin XuDewen Hu
May 9, 2015·IEEE Transactions on Neural Networks and Learning Systems·Tianheng SongKotaro Hirasawa
Jul 21, 2016·Computational Intelligence and Neuroscience·Chunyuan ZhangXinzheng Niu
Feb 2, 2016·IEEE Transactions on Neural Networks and Learning Systems·Xin XuHaibo He
Dec 2, 2015·IEEE Transactions on Neural Networks and Learning Systems·Fang WangJose C Principe
Feb 1, 2017·IEEE Transactions on Neural Networks and Learning Systems·Hongliang LiDing Wang
Jun 11, 2017·IEEE Transactions on Neural Networks and Learning Systems·Wentao GuoShengwei Mei
Sep 16, 2020·Journal of Physics. Condensed Matter : an Institute of Physics Journal·Edwin BedollaRamón Castañeda-Priego

Related Concepts

Knowledge Representation (Computer)
In Silico
Clinical Prediction Rule
Feedback - System Communication
Markov Chains
Psychological Reinforcement
Rietveld Refinement
Biomimetics

Trending Feeds

COVID-19

Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

Evolution of Pluripotency

Pluripotency refers to the ability of a cell to develop into three primary germ cell layers of the embryo. This feed focuses on the mechanisms that underlie the evolution of pluripotency. Here is the latest research.

Lipidomics & Rhinovirus Infection

Lipidomics can be used to examine the lipid species involved with pathogenic conditions, such as viral associated inflammation. Discovered the latest research on Lipidomics & Rhinovirus Infection.

Alzheimer's Disease: MS4A

Variants within the membrane-spanning 4-domains subfamily A (MS4A) gene cluster have recently been implicated in Alzheimer's disease in genome-wide association studies. Here is the latest research on Alzheimer's disease and MS4A.

Chronic Fatigue Syndrome

Chronic fatigue syndrome is a disease characterized by unexplained disabling fatigue; the pathology of which is incompletely understood. Discover the latest research on chronic fatigue syndrome here.

Torsion Dystonia

Torsion dystonia is a movement disorder characterized by loss of control of voluntary movements appearing as sustained muscle contractions and/or abnormal postures. Here is the latest research.

Generating Insulin-Secreting Cells

Reprogramming cells or using induced pluripotent stem cells to generate insulin-secreting cells has significant therapeutic implications for diabetics. Here is the latest research on generation of insulin-secreting cells.

Central Pontine Myelinolysis

Central Pontine Myelinolysis is a neurologic disorder caused most frequently by rapid correction of hyponatremia and is characterized by demyelination that affects the central portion of the base of the pons. Here is the latest research on this disease.

Epigenome Editing

Epigenome editing is the directed modification of epigenetic marks on chromatin at specified loci. This tool has many applications in research as well as in the clinic. Find the latest research on epigenome editing here.

Related Papers

IEEE Transactions on Neural Networks
D V Prokhorov, D C Wunsch
IEEE Transactions on Neural Networks and Learning Systems
Xin XuHaibo He
Neural Networks : the Official Journal of the International Neural Network Society
Paul J Werbos
Neural Networks : the Official Journal of the International Neural Network Society
Draguna Vrabie, Frank L Lewis
© 2021 Meta ULC. All rights reserved