Optimal Sensor Placement and Multimodal Fusion for Human Activity Recognition in Agricultural Tasks

被引:2
|
作者
Benos, Lefteris [1 ]
Tsaopoulos, Dimitrios [1 ]
Tagarakis, Aristotelis C. [1 ]
Kateris, Dimitrios [1 ]
Bochtis, Dionysis [1 ,2 ]
机构
[1] Ctr Res & Technol Hellas CERTH, Inst Bioecon & Agritechnol IBO, GR-57001 Thessaloniki, Greece
[2] FarmB Digital Agr, Doiraniis 17, Thessaloniki GR-54639, Greece
来源
APPLIED SCIENCES-BASEL | 2024年 / 14卷 / 18期
关键词
Long Short-Term Memory (LSTM) networks; wearable sensors; multi-sensor information fusion; human-robot collaboration; human factors; cost-optimal system configuration; ROBOTS; SPINE;
D O I
10.3390/app14188520
中图分类号
O6 [化学];
学科分类号
0703 ;
摘要
This study examines the impact of sensor placement and multimodal sensor fusion on the performance of a Long Short-Term Memory (LSTM)-based model for human activity classification taking place in an agricultural harvesting scenario involving human-robot collaboration. Data were collected from twenty participants performing six distinct activities using five wearable inertial measurement units placed at various anatomical locations. The signals collected from the sensors were first processed to eliminate noise and then input into an LSTM neural network for recognizing features in sequential time-dependent data. Results indicated that the chest-mounted sensor provided the highest F1-score of 0.939, representing superior performance over other placements and combinations of them. Moreover, the magnetometer surpassed the accelerometer and gyroscope, highlighting its superior ability to capture crucial orientation and motion data related to the investigated activities. However, multimodal fusion of accelerometer, gyroscope, and magnetometer data showed the benefit of integrating data from different sensor types to improve classification accuracy. The study emphasizes the effectiveness of strategic sensor placement and fusion in optimizing human activity recognition, thus minimizing data requirements and computational expenses, and resulting in a cost-optimal system configuration. Overall, this research contributes to the development of more intelligent, safe, cost-effective adaptive synergistic systems that can be integrated into a variety of applications.
引用
收藏
页数:16
相关论文
共 50 条
  • [1] Multimodal Sensor Data Fusion and Ensemble Modeling for Human Locomotion Activity Recognition
    Oh, Se Won
    Jeong, Hyuntae
    Chung, Seungeun
    Lim, Jeong Mook
    Noh, Kyoung Ju
    ADJUNCT PROCEEDINGS OF THE 2023 ACM INTERNATIONAL JOINT CONFERENCE ON PERVASIVE AND UBIQUITOUS COMPUTING & THE 2023 ACM INTERNATIONAL SYMPOSIUM ON WEARABLE COMPUTING, UBICOMP/ISWC 2023 ADJUNCT, 2023, : 546 - 550
  • [2] CNN-based Sensor Fusion Techniques for Multimodal Human Activity Recognition
    Muenzner, Sebastian
    Schmidt, Philip
    Reiss, Attila
    Hanselmann, Michael
    Stiefelhagen, Rainer
    Duerichen, Robert
    PROCEEDINGS OF THE 2017 ACM INTERNATIONAL SYMPOSIUM ON WEARABLE COMPUTERS (ISWC 17), 2017, : 158 - 165
  • [3] Unimodal and Multimodal Sensor Fusion for Wearable Activity Recognition
    Bello, Hymalai
    2024 IEEE INTERNATIONAL CONFERENCE ON PERVASIVE COMPUTING AND COMMUNICATIONS WORKSHOPS AND OTHER AFFILIATED EVENTS, PERCOM WORKSHOPS, 2024, : 364 - 365
  • [4] Sensor Data Acquisition and Multimodal Sensor Fusion for Human Activity Recognition Using Deep Learning
    Chung, Seungeun
    Lim, Jiyoun
    Noh, Kyoung Ju
    Kim, Gague
    Jeong, Hyuntae
    SENSORS, 2019, 19 (07)
  • [5] A Multimodal Fusion Approach for Human Activity Recognition
    Koutrintzes, Dimitrios
    Spyrou, Evaggelos
    Mathe, Eirini
    Mylonas, Phivos
    INTERNATIONAL JOURNAL OF NEURAL SYSTEMS, 2023, 33 (01)
  • [6] Optimal Sensor Placement for Human Activity Recognition with a Minimal Smartphone-IMU Setup
    Rahn, Vincent Xeno
    Zhou, Lin
    Klieme, Eric
    Arnrich, Bert
    PROCEEDINGS OF THE 10TH INTERNATIONAL CONFERENCE ON SENSOR NETWORKS (SENSORNETS), 2021, : 37 - 48
  • [7] Centaur: Robust Multimodal Fusion for Human Activity Recognition
    Xaviar, Sanju
    Yang, Xin
    Ardakanian, Omid
    IEEE SENSORS JOURNAL, 2024, 24 (11) : 18578 - 18591
  • [8] A Real-time Human Pose Estimation Approach for Optimal Sensor Placement in Sensor-based Human Activity Recognition
    Konak, Orhan
    Wischmann, Alexander
    van de Water, Robin
    Arnrich, Bert
    PROCEEDINGS OF THE 8TH INTERNATIONAL WORKSHOP ON SENSOR-BASED ACTIVITY RECOGNITION AND ARTIFICIAL INTELLIGENCE, IWOAR 2023, 2023,
  • [9] Sensor placement for optimal aerodynamic data fusion
    Barklage, Alexander
    Stradtner, Mario
    Bekemeyer, Philipp
    AEROSPACE SCIENCE AND TECHNOLOGY, 2024, 155
  • [10] Study regarding the optimal sensors placement on the body for human activity recognition
    Orha, I.
    Oniga, S.
    2014 IEEE 20TH INTERNATIONAL SYMPOSIUM FOR DESIGN AND TECHNOLOGY IN ELECTRONIC PACKAGING (SIITME), 2014, : 203 - 206