Optimal Sensor Placement and Multimodal Fusion for Human Activity Recognition in Agricultural Tasks

被引:2
|
作者
Benos, Lefteris [1 ]
Tsaopoulos, Dimitrios [1 ]
Tagarakis, Aristotelis C. [1 ]
Kateris, Dimitrios [1 ]
Bochtis, Dionysis [1 ,2 ]
机构
[1] Ctr Res & Technol Hellas CERTH, Inst Bioecon & Agritechnol IBO, GR-57001 Thessaloniki, Greece
[2] FarmB Digital Agr, Doiraniis 17, Thessaloniki GR-54639, Greece
来源
APPLIED SCIENCES-BASEL | 2024年 / 14卷 / 18期
关键词
Long Short-Term Memory (LSTM) networks; wearable sensors; multi-sensor information fusion; human-robot collaboration; human factors; cost-optimal system configuration; ROBOTS; SPINE;
D O I
10.3390/app14188520
中图分类号
O6 [化学];
学科分类号
0703 ;
摘要
This study examines the impact of sensor placement and multimodal sensor fusion on the performance of a Long Short-Term Memory (LSTM)-based model for human activity classification taking place in an agricultural harvesting scenario involving human-robot collaboration. Data were collected from twenty participants performing six distinct activities using five wearable inertial measurement units placed at various anatomical locations. The signals collected from the sensors were first processed to eliminate noise and then input into an LSTM neural network for recognizing features in sequential time-dependent data. Results indicated that the chest-mounted sensor provided the highest F1-score of 0.939, representing superior performance over other placements and combinations of them. Moreover, the magnetometer surpassed the accelerometer and gyroscope, highlighting its superior ability to capture crucial orientation and motion data related to the investigated activities. However, multimodal fusion of accelerometer, gyroscope, and magnetometer data showed the benefit of integrating data from different sensor types to improve classification accuracy. The study emphasizes the effectiveness of strategic sensor placement and fusion in optimizing human activity recognition, thus minimizing data requirements and computational expenses, and resulting in a cost-optimal system configuration. Overall, this research contributes to the development of more intelligent, safe, cost-effective adaptive synergistic systems that can be integrated into a variety of applications.
引用
收藏
页数:16
相关论文
共 50 条
  • [21] Action Recognition in Manufacturing Assembly using Multimodal Sensor Fusion
    Al-Amin, Md.
    Tao, Wenjin
    Doell, David
    Lingard, Ravon
    Yin, Zhaozheng
    Leu, Ming C.
    Qin, Ruwen
    25TH INTERNATIONAL CONFERENCE ON PRODUCTION RESEARCH MANUFACTURING INNOVATION: CYBER PHYSICAL MANUFACTURING, 2019, 39 : 158 - 167
  • [22] Image sensor fusion for multimodal biometric recognition in mobile devices
    Bhuvana, J.
    Barve, Amit
    Pradeep Kumar, Shah
    Dikshit, Sukanya
    Measurement: Sensors, 2024, 36
  • [23] DAMUN: A Domain Adaptive Human Activity Recognition Network Based on Multimodal Feature Fusion
    Feng, Xinxin
    Weng, Yuxin
    Li, Wenlong
    Chen, Pengcheng
    Zheng, Haifeng
    IEEE SENSORS JOURNAL, 2023, 23 (18) : 22019 - 22030
  • [24] Review on Multimodal Fusion Techniques for Human Emotion Recognition
    Karani, Ruhina
    Desai, Sharmishta
    INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2022, 13 (10) : 287 - 296
  • [25] Multimodal information fusion based human movement recognition
    Yao Shu
    Heng Zhang
    Multimedia Tools and Applications, 2020, 79 : 5043 - 5052
  • [26] Multimodal information fusion based human movement recognition
    Shu, Yao
    Zhang, Heng
    MULTIMEDIA TOOLS AND APPLICATIONS, 2020, 79 (7-8) : 5043 - 5052
  • [27] Wearable Sensor based Multimodal Human Activity Recognition Exploiting the Diversity of Classifier Ensemble
    Guo, Haodong
    Chen, Ling
    Peng, Liangying
    Chen, Gencai
    UBICOMP'16: PROCEEDINGS OF THE 2016 ACM INTERNATIONAL JOINT CONFERENCE ON PERVASIVE AND UBIQUITOUS COMPUTING, 2016, : 1112 - 1123
  • [28] Human Activity Recognition System Using Multimodal Sensor and Deep Learning Based on LSTM
    Shin, Soo-Yeun
    Cha, Joo-Heon
    TRANSACTIONS OF THE KOREAN SOCIETY OF MECHANICAL ENGINEERS A, 2018, 42 (02) : 111 - 121
  • [29] An Optimal Feature Selection Method for Human Activity Recognition Using Multimodal Sensory Data
    Haider, Tazeem
    Khan, Muhammad Hassan
    Farid, Muhammad Shahid
    INFORMATION, 2024, 15 (10)
  • [30] Adaptive multiple classifiers fusion for inertial sensor based human activity recognition
    Tian, Yiming
    Wang, Xitai
    Chen, Wei
    Liu, Zuojun
    Li, Lifeng
    CLUSTER COMPUTING-THE JOURNAL OF NETWORKS SOFTWARE TOOLS AND APPLICATIONS, 2019, 22 (Suppl 4): : S8141 - S8154