Efficient deep learning models based on tension techniques for sign language recognition

被引:2
|
作者
Attia, Nehal F. [1 ,2 ]
Ahmed, Mohamed T. Faheem Said [1 ]
Alshewimy, Mahmoud A. M. [1 ]
机构
[1] Tanta Univ, Fac Engn, Comp & Automat Control Dept, Tanta, Egypt
[2] Pharos Univ, Fac Engn, Comp Engn Dept, Alexandria, Egypt
来源
关键词
American sign language (ASL); YOLOv5; Object recognition; Computer vision; Convolutional block attention module (CBAM); Squeeze-and-excitation (SE); NEURAL-NETWORKS;
D O I
10.1016/j.iswa.2023.200284
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Communication by speaking prevails among the various ways of self-expression and communication between people. Speech presents a significant challenge for some disabled people, such as deaf people, deaf and hard of hearing, dumb and wordless persons. Therefore, these people rely on sign language to interact with others. Sign language is a system of movements and visual messages that ensure the integration of these individuals into groups that communicate vocally. On the other side, it is necessary to understand these individuals' gestures and linguistic semantics. The main objective of this work is to establish a new model that enhances the performance of the existing paradigms used for sign language recognition. This study developed three improved deep-learning models based on YOLOv5x and attention methods for recognizing the alphabetic and numeric information hand gestures convey. These models were evaluated using the MU HandImages ASL and OkkhorNama: BdSL datasets. The proposed models exceed those found in the literature, where the accuracy reached 98.9 % and 97.6 % with the MU HandImages ASL dataset and the OkkhorNama: BdSL dataset, respectively. The proposed models are light and fast enough to be used in real-time ASL recognition and to be deployed on any edge-based platform.
引用
收藏
页数:21
相关论文
共 50 条
  • [21] Isolated Sign Language Recognition Using Deep Learning
    Das, Sukanya
    Yadav, Sumit Kumar
    Samanta, Debasis
    COMPUTER VISION AND IMAGE PROCESSING, CVIP 2023, PT I, 2024, 2009 : 343 - 356
  • [22] Deep Learning Techniques for Spanish Sign Language Interpretation
    Martinez-Martin, Ester
    Morillas-Espejo, Francisco
    COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE, 2021, 2021
  • [23] Kurdish Sign Language Recognition Using Pre-Trained Deep Learning Models
    Alsaud, Ali A.
    Yousif, Raghad Z.
    Aziz, Marwan. M.
    Kareem, Shahab W.
    Maho, Amer J.
    JOURNAL OF ELECTRICAL SYSTEMS, 2024, 20 (06) : 1334 - 1344
  • [24] Efhamni: A Deep Learning-Based Saudi Sign Language Recognition Application
    Al Khuzayem, Lama
    Shafi, Suha
    Aljahdali, Safia
    Alkhamesie, Rawan
    Alzamzami, Ohoud
    SENSORS, 2024, 24 (10)
  • [25] Automated Arabic Sign Language Recognition System Based on Deep Transfer Learning
    Shahid, A., I
    Almotairi, Sultan
    INTERNATIONAL JOURNAL OF COMPUTER SCIENCE AND NETWORK SECURITY, 2019, 19 (10): : 144 - 152
  • [26] Deep learning-based sign language recognition system for static signs
    Wadhawan, Ankita
    Kumar, Parteek
    NEURAL COMPUTING & APPLICATIONS, 2020, 32 (12): : 7957 - 7968
  • [27] Deep Learning-Based Sign Language Recognition System for Cognitive Development
    Jebali, Maher
    Dakhli, Abdesselem
    Bakari, Wided
    COGNITIVE COMPUTATION, 2023, 15 (06) : 2189 - 2201
  • [28] Hand Landmark-Based Sign Language Recognition Using Deep Learning
    John, Jerry
    Sherif, Bismin, V
    MACHINE LEARNING AND AUTONOMOUS SYSTEMS, 2022, 269 : 147 - 157
  • [29] Survey on sign language recognition in context of vision-based and deep learning
    Subburaj, S.
    Murugavalli, S.
    Measurement: Sensors, 2022, 23
  • [30] A Comprehensive Study on Deep Learning-Based Methods for Sign Language Recognition
    Adaloglou, Nikolas
    Chatzis, Theocharis
    Papastratis, Ilias
    Stergioulas, Andreas
    Papadopoulos, Georgios Th.
    Zacharopoulou, Vassia
    Xydopoulos, George J.
    Atzakas, Klimnis
    Papazachariou, Dimitris
    Daras, Petros
    IEEE TRANSACTIONS ON MULTIMEDIA, 2022, 24 : 1750 - 1762