Joint intent detection and slot filling with syntactic and semantic features using multichannel CNN-BiLSTM

被引：0
|
作者：
Muhammad, Yusuf Idris [1 ]
Salim, Naomie [1 ]
Zainal, Anazida [1 ]
机构：
[1] Faculty of Computing, Universiti Teknologi Malaysia, Johor, Skudai, Malaysia
来源：
PeerJ Computer Science | 2024年 / 10卷
关键词：
Understanding spoken language is crucial for conversational agents; with intent detection and slot filling being the primary tasks in natural language understanding (NLU). Enhancing the NLU tasks can lead to an accurate and efficient virtual assistant thereby reducing the need for human intervention and expanding their applicability in other domains. Traditionally; these tasks have been addressed individually; but recent studies have highlighted their interconnection; suggesting better results when solved together. Recent advances in natural language processing have shown that pretrained word embeddings can enhance text representation and improve the generalization capabilities of models. However; the challenge of poor generalization in joint learning models for intent detection and slot filling remains due to limited annotated datasets. Additionally; traditional models face difficulties in capturing both the semantic and syntactic nuances of language; which are vital for accurate intent detection and slot filling. This study proposes a hybridized text representation method using a multichannel convolutional neural network with three embedding channels: non-contextual embeddings for semantic information; part-of-speech (POS) tag embeddings for syntactic features; and contextual embeddings for deeper contextual understanding. Specifically; we utilized word2vec for non-contextual embeddings; one-hot vectors for POS tags; and bidirectional encoder representations from transformers (BERT) for contextual embeddings. These embeddings are processed through a convolutional layer and a shared bidirectional long short-term memory (BiLSTM) network; followed by two softmax functions for intent detection and slot filling. Experiments on the air travel information system (ATIS) and SNIPS datasets demonstrated that our model significantly outperformed the baseline models; achieving an intent accuracy of 97.90% and slot filling F1-score of 98.86% on the ATIS dataset; and an intent accuracy of 98.88% and slot filling F1-score of 97.07% on the SNIPS dataset. These results highlight the effectiveness of our proposed approach in advancing dialogue systems; and paving the way for more accurate and efficient natural language understanding in real-world applications. © (2024); (PeerJ Inc.). All rights reserved;
D O I：
10.7717/PEERJ-CS.2346
中图分类号：
学科分类号：
摘要：
引用
收藏
相关论文
共 50 条
  • [41] Earthquake Magnitude Prediction using Spatia-Temporal Features Learning Based on Hybrid CNN-BiLSTM Model
    Kavianpour, Parisa
    Kavianpour, Mohammadreza
    Jahani, Ehsan
    Ramezani, Amin
    Proceedings - 2021 7th International Conference on Signal Processing and Intelligent Systems, ICSPIS 2021, 2021,
  • [42] Towards Explainable Joint Models via Information Theory for Multiple Intent Detection and Slot Filling
    Zhuang, Xianwei
    Cheng, Xuxin
    Zou, Yuexian
    THIRTY-EIGHTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 38 NO 17, 2024, : 19786 - 19794
  • [43] AGIF: An Adaptive Graph-Interactive Framework for Joint Multiple Intent Detection and Slot Filling
    Qin, Libo
    Xu, Xiao
    Che, Wanxiang
    Liu, Ting
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, EMNLP 2020, 2020, : 1807 - 1816
  • [44] Focus on Interaction: A Novel Dynamic Graph Model for Joint Multiple Intent Detection and Slot Filling
    Ding, Zeyuan
    Yang, Zhihao
    Lin, Hongfei
    Wang, Jian
    PROCEEDINGS OF THE THIRTIETH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, IJCAI 2021, 2021, : 3801 - 3807
  • [45] Reliable social media framework: fake news detection using modified feature attention based CNN-BiLSTM
    Srikanth, D.
    Prasad, K. Krishna
    Kannan, M.
    Kanchana, D.
    INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS, 2024,
  • [46] Promoting Unified Generative Framework with Descriptive Prompts for Joint Multi-Intent Detection and Slot Filling
    Ma, Zhiyuan
    Qin, Jiwei
    Pan, Meiqi
    Tang, Song
    Mi, Jinpeng
    Liu, Dan
    ELECTRONICS, 2024, 13 (06)
  • [47] LAGIM: A Label-Aware Graph Interaction Model for Joint Multiple Intent Detection and Slot Filling
    Li, Penghua
    Huang, Ziheng
    2023 35TH CHINESE CONTROL AND DECISION CONFERENCE, CCDC, 2023, : 448 - 453
  • [48] End-to-end masked graph-based CRF for joint slot filling and intent detection
    Tang, Hao
    Ji, Donghong
    Zhou, Qiji
    NEUROCOMPUTING, 2020, 413 (413) : 348 - 359
  • [49] Learning to Bridge Metric Spaces: Few-shot Joint Learning of Intent Detection and Slot Filling
    Hou, Yutai
    Lai, Yongkui
    Chen, Cheng
    Che, Wanxiang
    Liu, Ting
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, ACL-IJCNLP 2021, 2021, : 3190 - 3200
  • [50] A multi-dimensional hybrid CNN-BiLSTM framework for epileptic seizure detection using electroencephalogram signal scrutiny
    Britto, K. R. Aravind
    Srinivasan, Saravanan
    Mathivanan, Sandeep Kumar
    Venkatesan, Muthukumaran
    Malar, M. B. Benjula Anbu
    Mallik, Saurav
    Qin, Hong
    SYSTEMS AND SOFT COMPUTING, 2023, 5
12345