Enhancing Visual Feedback Control through Early Fusion Deep Learning

被引:0
|
作者
Botezatu, Adrian-Paul [1 ]
Ferariu, Lavinia-Eugenia [1 ]
Burlacu, Adrian [1 ]
机构
[1] Gheorghe Asachi Tech Univ Iasi, Fac Automatic Control & Comp Engn, D Mangeron 27, Iasi 700050, Romania
关键词
visual feedback control; convolutional neural network; early fusion; segmentation; feature points; image moments; IMAGE MOMENTS; SERVO CONTROL; SET;
D O I
10.3390/e25101378
中图分类号
O4 [物理学];
学科分类号
0702 ;
摘要
A visual servoing system is a type of control system used in robotics that employs visual feedback to guide the movement of a robot or a camera to achieve a desired task. This problem is addressed using deep models that receive a visual representation of the current and desired scene, to compute the control input. The focus is on early fusion, which consists of using additional information integrated into the neural input array. In this context, we discuss how ready-to-use information can be directly obtained from the current and desired scenes, to facilitate the learning process. Inspired by some of the most effective traditional visual servoing techniques, we introduce early fusion based on image moments and provide an extensive analysis of approaches based on image moments, region-based segmentation, and feature points. These techniques are applied stand-alone or in combination, to allow obtaining maps with different levels of detail. The role of the extra maps is experimentally investigated for scenes with different layouts. The results show that early fusion facilitates a more accurate approximation of the linear and angular camera velocities, in order to control the movement of a 6-degree-of-freedom robot from a current configuration to a desired one. The best results were obtained for the extra maps providing details of low and medium levels.
引用
收藏
页数:26
相关论文
共 50 条
  • [1] Enhancing Deep Learning with Visual Interactions
    Krokos, Eric
    Cheng, Hsueh-Chen
    Chang, Jessica
    Nebesh, Bohdan
    Paul, Celeste Lyn
    Whitley, Kirsten
    Varshney, Amitabh
    ACM TRANSACTIONS ON INTERACTIVE INTELLIGENT SYSTEMS, 2019, 9 (01)
  • [2] Sensor Fusion for Robot Control through Deep Reinforcement Learning
    Bohez, Steven
    Verbelen, Tim
    De Coninck, Elias
    Vankeirsbilck, Bert
    Simoens, Pieter
    Dhoedt, Bart
    2017 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS), 2017, : 2365 - 2370
  • [3] Enhancing speech emotion recognition through deep learning and handcrafted feature fusion
    Eris, Fatma Gunes
    Akbal, Erhan
    APPLIED ACOUSTICS, 2024, 222
  • [4] Enhancing early Alzheimer's disease classification accuracy through the fusion of sMRI and rsMEG data: a deep learning approach
    Liu, Yuchen
    Wang, Ling
    Ning, Xiaolin
    Gao, Yang
    Wang, Defeng
    FRONTIERS IN NEUROSCIENCE, 2024, 18
  • [5] Enhancing the effectiveness of English grammar teaching through biomechanical feedback and deep learning algorithms
    Gong, Xueqin
    Li, Dongjie
    MCB Molecular and Cellular Biomechanics, 2024, 21 (03):
  • [6] Enhancing HVAC control systems through transfer learning with deep reinforcement learning agents
    Kadamala, Kevlyn
    Chambers, Des
    Barrett, Enda
    SMART ENERGY, 2024, 13
  • [7] Enhancing cervical cancer detection and robust classification through a fusion of deep learning models
    Mathivanan, Sandeep Kumar
    Francis, Divya
    Srinivasan, Saravanan
    Khatavkar, Vaibhav
    Karthikeyan, P.
    Shah, Mohd Asif
    SCIENTIFIC REPORTS, 2024, 14 (01):
  • [8] Enhancing Learning through Formative Assessment and Feedback
    Thamrin, Nur Sehang
    Fahri, Ardian
    ACADEMY OF MANAGEMENT LEARNING & EDUCATION, 2024, 23 (01) : 195 - 197
  • [9] Enhancing Learning through Formative Assessment and Feedback
    Levesque, Paul J.
    TEACHING THEOLOGY AND RELIGION, 2010, 13 (02): : 184 - 185
  • [10] Enhancing learning through formative assessment and feedback
    Baartman, Liesbeth
    BRITISH JOURNAL OF EDUCATIONAL TECHNOLOGY, 2008, 39 (05) : 957 - 957