Efficient deep learning models based on tension techniques for sign language recognition

被引:2
|
作者
Attia, Nehal F. [1 ,2 ]
Ahmed, Mohamed T. Faheem Said [1 ]
Alshewimy, Mahmoud A. M. [1 ]
机构
[1] Tanta Univ, Fac Engn, Comp & Automat Control Dept, Tanta, Egypt
[2] Pharos Univ, Fac Engn, Comp Engn Dept, Alexandria, Egypt
来源
关键词
American sign language (ASL); YOLOv5; Object recognition; Computer vision; Convolutional block attention module (CBAM); Squeeze-and-excitation (SE); NEURAL-NETWORKS;
D O I
10.1016/j.iswa.2023.200284
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Communication by speaking prevails among the various ways of self-expression and communication between people. Speech presents a significant challenge for some disabled people, such as deaf people, deaf and hard of hearing, dumb and wordless persons. Therefore, these people rely on sign language to interact with others. Sign language is a system of movements and visual messages that ensure the integration of these individuals into groups that communicate vocally. On the other side, it is necessary to understand these individuals' gestures and linguistic semantics. The main objective of this work is to establish a new model that enhances the performance of the existing paradigms used for sign language recognition. This study developed three improved deep-learning models based on YOLOv5x and attention methods for recognizing the alphabetic and numeric information hand gestures convey. These models were evaluated using the MU HandImages ASL and OkkhorNama: BdSL datasets. The proposed models exceed those found in the literature, where the accuracy reached 98.9 % and 97.6 % with the MU HandImages ASL dataset and the OkkhorNama: BdSL dataset, respectively. The proposed models are light and fast enough to be used in real-time ASL recognition and to be deployed on any edge-based platform.
引用
收藏
页数:21
相关论文
共 50 条
  • [1] Arabic Sign Language Recognition Using Deep Learning Models
    Al-Barham, Muhammad
    Abu Sa'aleek, Ahmad
    Al-Odat, Mohammad
    Hamad, Ghada
    Al-Yaman, Musa
    Elnagar, Ashraf
    2022 13TH INTERNATIONAL CONFERENCE ON INFORMATION AND COMMUNICATION SYSTEMS (ICICS), 2022, : 226 - 231
  • [2] Sign Language Recognition: A Comparative Analysis of Deep Learning Models
    Premkumar, Aswathi
    Krishna, R. Hridya
    Chanalya, Nikita
    Meghadev, C.
    Varma, Utkrist Arvind
    Anjali, T.
    Rani, S. Siji
    INVENTIVE COMPUTATION AND INFORMATION TECHNOLOGIES, ICICIT 2021, 2022, 336 : 1 - 13
  • [3] Review of Sign Language Recognition Based on Deep Learning
    Zhang Shujun
    Zhang Qun
    Li Hui
    JOURNAL OF ELECTRONICS & INFORMATION TECHNOLOGY, 2020, 42 (04) : 1021 - 1032
  • [4] A Deep Learning based Recognition System for Yemeni Sign Language
    Dabwan, Basel A.
    Jadhav, Mukti E.
    2021 INTERNATIONAL CONFERENCE OF MODERN TRENDS IN INFORMATION AND COMMUNICATION TECHNOLOGY INDUSTRY (MTICTI 2021), 2021, : 20 - 24
  • [5] Deep Learning for Sign Language Recognition: Current Techniques, Benchmarks, and Open Issues
    Al-Qurishi, Muhammad
    Khalid, Thariq
    Souissi, Riad
    IEEE ACCESS, 2021, 9 : 126917 - 126951
  • [6] Sign Language Recognition Using Deep Learning
    Ray, Anushka
    Syed, Shahbaz
    Poornima, S.
    Pushpalatha, M.
    JOURNAL OF PHARMACEUTICAL NEGATIVE RESULTS, 2022, 13 : 421 - 428
  • [7] Pakistan sign language recognition: leveraging deep learning models with limited dataset
    Hafiz Muhammad Hamza
    Aamir Wali
    Machine Vision and Applications, 2023, 34
  • [8] Pakistan sign language recognition: leveraging deep learning models with limited dataset
    Hamza, Hafiz Muhammad
    Wali, Aamir
    MACHINE VISION AND APPLICATIONS, 2023, 34 (05)
  • [9] Efficient Sign Language Recognition System and Dataset Creation Method Based on Deep Learning and Image Processing
    Cavalcante Carneiro, A. L.
    Silva, L. Brito
    Pinheiro Salvadeo, D. H.
    THIRTEENTH INTERNATIONAL CONFERENCE ON DIGITAL IMAGE PROCESSING (ICDIP 2021), 2021, 11878
  • [10] Deep Learning-Based Approach for Sign Language Gesture Recognition With Efficient Hand Gesture Representation
    Al-Hammadi, Muneer
    Muhammad, Ghulam
    Abdul, Wadood
    Alsulaiman, Mansour
    Bencherif, Mohammed A.
    Alrayes, Tareq S.
    Mathkour, Hassan
    Mekhtiche, Mohamed Amine
    IEEE ACCESS, 2020, 8 (08): : 192527 - 192542