Large Vocabulary SOUL Neural Network Language Models

被引：0
|
作者：
Le, Hai-Son [1 ,2 ]
Oparin, Ilya [2 ]
Messaoudi, Abdel [2 ]
Allauzen, Alexandre [1 ,2 ]
Gauvain, Jean-Luc [2 ]
Yvon, Francois [1 ,2 ]
机构：
[1] Univ Paris 11, BP 133, F-91403 Orsay, France
[2] Spoken Language Proc Grp, CNRS, LIMSI, F-91403 Orsay, France
来源：
12TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION 2011 (INTERSPEECH 2011), VOLS 1-5 | 2011年
关键词：
Neural Network Language Model; Automatic Speech Recognition; Speech-To-Text;
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
This paper presents continuation of research on Structured OUt-put Layer Neural Network language models (SOUL NNLM) for automatic speech recognition. As SOUL NNLMs allow estimating probabilities for all in-vocabulary words and not only for those pertaining to a limited shortlist, we investigate. its performance on a large-vocabulary task. Significant improvements both in perplexity and word error rate over conventional shortlist-based NNLMs are shown on a challenging Arabic GALE task characterized by a recognition vocabulary of about 300k entries. A new training scheme is proposed for SOUL NNLMs that is based on separate training of the out-of-shortlist part of the output layer. It enables using more data at each iteration of a neural network without any considerable slow-down in training and brings additional improvements in speech recognition performance.
引用
收藏
页码：1480 / +
页数：2
相关论文
共 50 条
  • [21] UNNORMALIZED EXPONENTIAL AND NEURAL NETWORK LANGUAGE MODELS
    Sethy, Abhinav
    Chen, Stanley
    Arisoy, Ebru
    Ramabhadran, Bhuvana
    2015 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING (ICASSP), 2015, : 5416 - 5420
  • [22] SCALING RECURRENT NEURAL NETWORK LANGUAGE MODELS
    Williams, Will
    Prasad, Niranjani
    Mrva, David
    Ash, Tom
    Robinson, Tony
    2015 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING (ICASSP), 2015, : 5391 - 5395
  • [23] Tracking Child Language Development With Neural Network Language Models
    Sagae, Kenji
    FRONTIERS IN PSYCHOLOGY, 2021, 12
  • [24] The Deep Tensor Neural Network With Applications to Large Vocabulary Speech Recognition
    Yu, Dong
    Deng, Li
    Seide, Frank
    IEEE TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING, 2013, 21 (02): : 388 - 396
  • [25] An approach to vocabulary expansion for neural network language model by means of hierarchical clustering
    Pavel, Dudarin
    Nadezhda, Yarushkina
    PROCEEDINGS OF THE 11TH CONFERENCE OF THE EUROPEAN SOCIETY FOR FUZZY LOGIC AND TECHNOLOGY (EUSFLAT 2019), 2019, 1 : 614 - 618
  • [26] A Batch Noise Contrastive Estimation Approach for Training Large Vocabulary Language Models
    Oualil, Youssef
    Klakow, Dietrich
    18TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2017), VOLS 1-6: SITUATED INTERACTION, 2017, : 264 - 268
  • [27] Free Acoustic and Language Models for Large Vocabulary Continuous Speech Recognition in Swedish
    Vanhainen, Niklas
    Salvi, Giampiero
    LREC 2014 - NINTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION, 2014,
  • [28] Modeling English vocabulary acquisition through the biomechanics of speech and Large Language Models
    Shang, Jingya
    MCB Molecular and Cellular Biomechanics, 2025, 22 (01):
  • [29] RECURRENT NEURAL NETWORK LANGUAGE MODELS FOR KEYWORD SEARCH
    Chen, X.
    Ragni, A.
    Vasilakes, J.
    Liu, X.
    Knill, K.
    Gales, M. J. F.
    2017 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2017, : 5775 - 5779
  • [30] Unsupervised Adaptation of Recurrent Neural Network Language Models
    Gangireddy, Siva Reddy
    Swietojanski, Pawel
    Bell, Peter
    Renals, Steve
    17TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2016), VOLS 1-5: UNDERSTANDING SPEECH PROCESSING IN HUMANS AND MACHINES, 2016, : 2333 - 2337
12345