Feb 6, 2008

LSTM recurrent networks learn simple context-free and context-sensitive languages

IEEE Transactions on Neural Networks
F Gers, E Schmidhuber

Abstract

Previous work on learning regular languages from exemplary training sequences showed that long short-term memory (LSTM) outperforms traditional recurrent neural networks (RNNs). We demonstrate LSTMs superior performance on context-free language benchmarks for RNNs, and show that it works even better than previous hardwired or highly specialized architectures. To the best of our knowledge, LSTM variants are also the first RNNs to learn a simple context-sensitive language, namely a(n)b(n)c(n).

  • References7
  • Citations31

Citations

Mentioned in this Paper

Neural Network Simulation

Trending Feeds

COVID-19

Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

Bone Marrow Neoplasms

Bone Marrow Neoplasms are cancers that occur in the bone marrow. Discover the latest research on Bone Marrow Neoplasms here.

IGA Glomerulonephritis

IgA glomerulonephritis is a chronic form of glomerulonephritis characterized by deposits of predominantly Iimmunoglobin A in the mesangial area. Discover the latest research on IgA glomerulonephritis here.

Cryogenic Electron Microscopy

Cryogenic electron microscopy (Cryo-EM) allows the determination of biological macromolecules and their assemblies at a near-atomic resolution. Here is the latest research.

STING Receptor Agonists

Stimulator of IFN genes (STING) are a group of transmembrane proteins that are involved in the induction of type I interferon that is important in the innate immune response. The stimulation of STING has been an active area of research in the treatment of cancer and infectious diseases. Here is the latest research on STING receptor agonists.

LRRK2 & Immunity During Infection

Mutations in the LRRK2 gene are a risk-factor for developing Parkinson’s disease. However, LRRK2 has been shown to function as a central regulator of vesicular trafficking, infection, immunity, and inflammation. Here is the latest research on the role of this kinase on immunity during infection.

Antiphospholipid Syndrome

Antiphospholipid syndrome or antiphospholipid antibody syndrome (APS or APLS), is an autoimmune, hypercoagulable state caused by the presence of antibodies directed against phospholipids.

Meningococcal Myelitis

Meningococcal myelitis is characterized by inflammation and myelin damage to the meninges and spinal cord. Discover the latest research on meningococcal myelitis here.

Alzheimer's Disease: MS4A

Variants within membrane-spanning 4-domains subfamily A (MS4A) gene cluster have recently been implicated in Alzheimer's disease by recent genome-wide association studies. Here is the latest research.

Related Papers

IEEE Transactions on Neural Networks
M Boden, J Wiles
Neural Networks : the Official Journal of the International Neural Network Society
Juan Antonio Pérez-OrtizJürgen Schmidhuber
Neural Networks : the Official Journal of the International Neural Network Society
Derek Monner, James A Reggia
© 2020 Meta ULC. All rights reserved