Joint intent detection and slot filling with syntactic and semantic features using multichannel CNN-BiLSTM

被引：0
|
作者：
Muhammad, Yusuf Idris [1 ]
Salim, Naomie [1 ]
Zainal, Anazida [1 ]
机构：
[1] Faculty of Computing, Universiti Teknologi Malaysia, Johor, Skudai, Malaysia
来源：
PeerJ Computer Science | 2024年 / 10卷
关键词：
Understanding spoken language is crucial for conversational agents; with intent detection and slot filling being the primary tasks in natural language understanding (NLU). Enhancing the NLU tasks can lead to an accurate and efficient virtual assistant thereby reducing the need for human intervention and expanding their applicability in other domains. Traditionally; these tasks have been addressed individually; but recent studies have highlighted their interconnection; suggesting better results when solved together. Recent advances in natural language processing have shown that pretrained word embeddings can enhance text representation and improve the generalization capabilities of models. However; the challenge of poor generalization in joint learning models for intent detection and slot filling remains due to limited annotated datasets. Additionally; traditional models face difficulties in capturing both the semantic and syntactic nuances of language; which are vital for accurate intent detection and slot filling. This study proposes a hybridized text representation method using a multichannel convolutional neural network with three embedding channels: non-contextual embeddings for semantic information; part-of-speech (POS) tag embeddings for syntactic features; and contextual embeddings for deeper contextual understanding. Specifically; we utilized word2vec for non-contextual embeddings; one-hot vectors for POS tags; and bidirectional encoder representations from transformers (BERT) for contextual embeddings. These embeddings are processed through a convolutional layer and a shared bidirectional long short-term memory (BiLSTM) network; followed by two softmax functions for intent detection and slot filling. Experiments on the air travel information system (ATIS) and SNIPS datasets demonstrated that our model significantly outperformed the baseline models; achieving an intent accuracy of 97.90% and slot filling F1-score of 98.86% on the ATIS dataset; and an intent accuracy of 98.88% and slot filling F1-score of 97.07% on the SNIPS dataset. These results highlight the effectiveness of our proposed approach in advancing dialogue systems; and paving the way for more accurate and efficient natural language understanding in real-world applications. © (2024); (PeerJ Inc.). All rights reserved;
D O I：
10.7717/PEERJ-CS.2346
中图分类号：
学科分类号：
摘要：
引用
收藏
相关论文
共 50 条
  • [31] A Two-Stage Selective Fusion Framework for Joint Intent Detection and Slot Filling
    Ma, Ziyu
    Sun, Bin
    Li, Shutao
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2024, 35 (03) : 3874 - 3885
  • [32] SlotRefine: A Fast Non-Autoregressive Model for Joint Intent Detection and Slot Filling
    Wu, Di
    Ding, Liang
    Lu, Fan
    Xie, Jian
    PROCEEDINGS OF THE 2020 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP), 2020, : 1932 - 1937
  • [33] Joint Training Model of Intent Detection and Slot Filling for Multi Granularity Implicit Guidance
    Li, Bin
    Wang, Weihua
    Bao, Feilong
    2022 INTERNATIONAL CONFERENCE ON ASIAN LANGUAGE PROCESSING (IALP 2022), 2022, : 271 - 274
  • [34] A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling
    E, Haihong
    Niu, Peiqing
    Chen, Zhongfu
    Song, Meina
    57TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2019), 2019, : 5467 - 5471
  • [35] Joint Intent Detection and Slot Filling of Knowledge Question Answering for Agricultural Diseases and Pests
    Guo X.
    Hao X.
    Yao X.
    Li L.
    Nongye Jixie Xuebao/Transactions of the Chinese Society for Agricultural Machinery, 2023, 54 (01): : 205 - 215
  • [36] CONVOLUTIONAL NEURAL NETWORK BASED TRIANGULAR CRF FOR JOINT INTENT DETECTION AND SLOT FILLING
    Xu, Puyang
    Sarikaya, Ruhi
    2013 IEEE WORKSHOP ON AUTOMATIC SPEECH RECOGNITION AND UNDERSTANDING (ASRU), 2013, : 78 - 83
  • [37] Joint agricultural intent detection and slot filling based on enhanced heterogeneous attention mechanism
    Hao, Xia
    Wang, Lu
    Zhu, Hongmei
    Guo, Xuchao
    COMPUTERS AND ELECTRONICS IN AGRICULTURE, 2023, 207
  • [38] An optimised Darknet traffic detection system using modified locally connected CNN-BiLSTM network
    Shaikh, Abdullah Abdul Sattar
    Bhargavi, M. S.
    Kumar, C. Pavan
    INTERNATIONAL JOURNAL OF AD HOC AND UBIQUITOUS COMPUTING, 2023, 43 (02) : 87 - 96
  • [39] Sleep Apnea Detection From Variational Mode Decomposed EEG Signal Using a Hybrid CNN-BiLSTM
    Mahmud, Tanvir
    Khan, Ishtiaque Ahmed
    Mahmud, Talha Ibn
    Fattah, Shaikh Anowarul
    Zhu, Wei-Ping
    Ahmad, M. Omair
    IEEE ACCESS, 2021, 9 : 102355 - 102367
  • [40] Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling
    Liu, Bing
    Lane, Ian
    17TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2016), VOLS 1-5: UNDERSTANDING SPEECH PROCESSING IN HUMANS AND MACHINES, 2016, : 685 - 689
12345