LSTM recurrent networks learn simple context-free and context-sensitive languages

被引：467

作者：

Gers, FA ^{[1
]}

Schtmidhuber, J ^{[1
]}

机构：

[1] IDSIA, CH-6928 Manno, Switzerland

来源：

IEEE TRANSACTIONS ON NEURAL NETWORKS | 2001年 / 12卷 / 06期

关键词：

context-free languages (CFLs); context-sensitive languages (CSLs); long short-term memory (LSTM); recurrent neural networks (RNNs);

D O I：

10.1109/72.963769

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

Previous work on learning regular languages from exemplary training sequences showed that long short-term memory (LSTM) outperforms traditional recurrent neural networks (RNNs). Here we demonstrate LSTMs superior performance on context-free language (CFL) benchmarks for RNNs, and show that it works even better than previous hardwired or highly specialized architectures. To the best of our knowledge, LSTM variants are also the first RNNs to learn a simple context-sensitive language (CSL), namely a(n)b(n)c(n).

引用

页码：1333 / 1340

页数：8

共 32 条

[1]

[Anonymous], PROCEEDINGS OF THE S

[2] LEARNING LONG-TERM DEPENDENCIES WITH GRADIENT DESCENT IS DIFFICULT [J].