Efficient deep learning models based on tension techniques for sign language recognition

被引:2
|
作者
Attia, Nehal F. [1 ,2 ]
Ahmed, Mohamed T. Faheem Said [1 ]
Alshewimy, Mahmoud A. M. [1 ]
机构
[1] Tanta Univ, Fac Engn, Comp & Automat Control Dept, Tanta, Egypt
[2] Pharos Univ, Fac Engn, Comp Engn Dept, Alexandria, Egypt
来源
关键词
American sign language (ASL); YOLOv5; Object recognition; Computer vision; Convolutional block attention module (CBAM); Squeeze-and-excitation (SE); NEURAL-NETWORKS;
D O I
10.1016/j.iswa.2023.200284
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Communication by speaking prevails among the various ways of self-expression and communication between people. Speech presents a significant challenge for some disabled people, such as deaf people, deaf and hard of hearing, dumb and wordless persons. Therefore, these people rely on sign language to interact with others. Sign language is a system of movements and visual messages that ensure the integration of these individuals into groups that communicate vocally. On the other side, it is necessary to understand these individuals' gestures and linguistic semantics. The main objective of this work is to establish a new model that enhances the performance of the existing paradigms used for sign language recognition. This study developed three improved deep-learning models based on YOLOv5x and attention methods for recognizing the alphabetic and numeric information hand gestures convey. These models were evaluated using the MU HandImages ASL and OkkhorNama: BdSL datasets. The proposed models exceed those found in the literature, where the accuracy reached 98.9 % and 97.6 % with the MU HandImages ASL dataset and the OkkhorNama: BdSL dataset, respectively. The proposed models are light and fast enough to be used in real-time ASL recognition and to be deployed on any edge-based platform.
引用
收藏
页数:21
相关论文
共 50 条
  • [41] Deep Learning for American Sign Language Fingerspelling Recognition System
    Nguyen, Huy B. D.
    Hung Ngoc Do
    2019 26TH INTERNATIONAL CONFERENCE ON TELECOMMUNICATIONS (ICT), 2019, : 314 - 318
  • [42] Traffic sign recognition based on deep learning
    Zhu, Yanzhao
    Yan, Wei Qi
    MULTIMEDIA TOOLS AND APPLICATIONS, 2022, 81 (13) : 17779 - 17791
  • [43] Deepsign: Sign Language Detection and Recognition Using Deep Learning
    Kothadiya, Deep
    Bhatt, Chintan
    Sapariya, Krenil
    Patel, Kevin
    Gil-Gonzalez, Ana-Belen
    Corchado, Juan M.
    ELECTRONICS, 2022, 11 (11)
  • [44] SignBERT: A BERT-Based Deep Learning Framework for Continuous Sign Language Recognition
    Zhou, Zhenxing
    Tam, Vincent W. L.
    Lam, Edmund Y.
    IEEE ACCESS, 2021, 9 : 161669 - 161682
  • [45] Vision Based Deep Learning Approach for Dynamic Indian Sign Language Recognition in Healthcare
    Uchil, Aditya P.
    Jha, Smriti
    Sudha, B. G.
    COMPUTATIONAL VISION AND BIO-INSPIRED COMPUTING, 2020, 1108 : 371 - 383
  • [46] Deep Learning-Based Sign Language Recognition for Hearing and Speaking Impaired People
    Alnfiai, Mrim M.
    INTELLIGENT AUTOMATION AND SOFT COMPUTING, 2023, 36 (02): : 1653 - 1669
  • [47] A Critical Study of Recent Deep Learning-Based Continuous Sign Language Recognition
    Hanan A. Taher
    Subhi R. M. Zeebaree
    The Review of Socionetwork Strategies, 2025, 19 (1) : 131 - 161
  • [48] Manual and non-manual sign language recognition framework using hybrid deep learning techniques
    Javaid, Sameena
    Rizvi, Safdar
    JOURNAL OF INTELLIGENT & FUZZY SYSTEMS, 2023, 45 (03) : 3823 - 3833
  • [49] A deep sign language recognition system for Indian sign language
    Das, Soumen
    Biswas, Saroj Kr
    Purkayastha, Biswajit
    NEURAL COMPUTING & APPLICATIONS, 2023, 35 (02): : 1469 - 1481
  • [50] A deep sign language recognition system for Indian sign language
    Soumen Das
    Saroj Kr. Biswas
    Biswajit Purkayastha
    Neural Computing and Applications, 2023, 35 : 1469 - 1481