Apr 9, 2020

Developing a novel recurrent neural network architecture with fewer parameters and good learning performance

BioRxiv : the Preprint Server for Biology
Kazunori D Yamada

Abstract

Recurrent neural networks (RNNs) are among the most promising of the many artificial intelligence techniques now under development, showing great potential for memory, interaction, and linguistic understanding. Among the more sophisticated RNNs are long short-term memory (LSTM) and gated recurrent units (GRUs), which emulate animal brain behavior; these methods yield superior memory and learning speed because of the excellent core structure of their architectures. In this study, we attempted to make further improvements in core structure and develop a novel, compact architecture with a high learning speed. We stochastically generated 30000 RNN architectures, evaluated their performance, and selected the one most capable of memorizing long contexts with relatively few parameters. This RNN, YamRNN, had fewer parameters than LSTM and GRU by a factor of two-thirds or better and reduced the time required to achieve the same learning performance on a sequence classification task as LSTM and GRU by 80% at maximum. This novel RNN architecture is expected to be useful for addressing problems such as predictions and analyses on contextual data and also suggests that there is room for the development of better architectures.

  • References
  • Citations

References

  • We're still populating references for this paper, please check back later.
  • References
  • Citations

Citations

  • This paper may not have been cited yet.

Mentioned in this Paper

Dysequilibrium Syndrome
Glycosylphosphatidylinositol Linkage
Sequencing
Genome Sequencing
Whole Genome Sequencing
Genetic Linkage
Genotype Determination
Copy Number Polymorphism
Cohort

Related Feeds

BioRxiv & MedRxiv Preprints

BioRxiv and MedRxiv are the preprint servers for biology and health sciences respectively, operated by Cold Spring Harbor Laboratory. Here are the latest preprint articles (which are not peer-reviewed) from BioRxiv and MedRxiv.

Related Papers

Conference Proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society
Ali Samadani
Beijing da xue xue bao. Yi xue ban = Journal of Peking University. Health sciences
W TangY S Wang
© 2020 Meta ULC. All rights reserved