Using human gaze in few-shot imitation learning for robot manipulation

被引:1
|
作者
Hamano, Shogo [1 ]
Kim, Heecheol [1 ]
Ohmura, Yoshiyuki [1 ]
Kuniyoshi, Yasuo [1 ]
机构
[1] Univ Tokyo, Grad Sch Informat Sci & Technol, Lab Intelligent Syst & Informat, Bunkyo Ku, 7-3-1 Hongo, Tokyo, Japan
来源
2022 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS) | 2022年
关键词
Imitation Learning; Deep Learning in Grasping and Manipulation; Few-shot Learning; Meta-learning; Telerobotics and Teleoperation;
D O I
10.1109/IROS47612.2022.9981706
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Imitation learning has attracted attention as a method for realizing complex robot control without programmed robot behavior. Meta-imitation learning has been proposed to solve the high cost of data collection and low generalizability to new tasks that imitation learning suffers from. Meta-imitation can learn new tasks involving unknown objects from a small amount of data by learning multiple tasks during training. However, meta-imitation learning, especially using images, is still vulnerable to changes in the background, which occupies a large portion of the input image. This study introduces a human gaze into meta-imitation learning-based robot control. We created a model with model-agnostic meta-learning to predict the gaze position from the image by measuring the gaze with an eye tracker in the head-mounted display. Using images around the predicted gaze position as an input makes the model robust to changes in visual information. We experimentally verified the performance of the proposed method through picking tasks using a simulated robot. The results indicate that our proposed method has a greater ability than the conventional method to learn a new task from only 9 demonstrations even if the object's color or the background pattern changes between the training and test.
引用
收藏
页码:8622 / 8629
页数:8
相关论文
共 50 条
  • [31] Robot Manipulation Learning Using Generative Adversarial Imitation Learning
    Jabri, Mohamed Khalil
    PROCEEDINGS OF THE THIRTIETH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, IJCAI 2021, 2021, : 4893 - 4894
  • [32] RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitation
    Vecerik, Mel
    Doersch, Carl
    Yang, Yi
    Davchev, Todor
    Aytar, Yusuf
    Zhou, Guangyao
    Hadsell, Raia
    Agapito, Lourdes
    Scholz, Jon
    2024 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION, ICRA 2024, 2024, : 5397 - 5403
  • [33] Co-Learning for Few-Shot Learning
    Rui Xu
    Lei Xing
    Shuai Shao
    Baodi Liu
    Kai Zhang
    Weifeng Liu
    Neural Processing Letters, 2022, 54 : 3339 - 3356
  • [34] RankDNN: Learning to Rank for Few-Shot Learning
    Guo, Qianyu
    Gong Haotong
    Wei, Xujun
    Fu, Yanwei
    Yu, Yizhou
    Zhang, Wenqiang
    Ge, Weifeng
    THIRTY-SEVENTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 37 NO 1, 2023, : 728 - 736
  • [35] Learning about few-shot concept learning
    Ananya Rastogi
    Nature Computational Science, 2022, 2 : 698 - 698
  • [36] Co-Learning for Few-Shot Learning
    Xu, Rui
    Xing, Lei
    Shao, Shuai
    Liu, Baodi
    Zhang, Kai
    Liu, Weifeng
    NEURAL PROCESSING LETTERS, 2022, 54 (04) : 3339 - 3356
  • [37] DemoGrasp: Few-Shot Learning for Robotic Grasping with Human Demonstration
    Wang, Pengyuan
    Manhardt, Fabian
    Minciullo, Luca
    Garattoni, Lorenzo
    Meier, Sven
    Navab, Nassir
    Busam, Benjamin
    2021 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS), 2021, : 5733 - 5740
  • [38] Federated Few-Shot Learning with Adversarial Learning
    Fan, Chenyou
    Huang, Jianwei
    2021 19TH INTERNATIONAL SYMPOSIUM ON MODELING AND OPTIMIZATION IN MOBILE, AD HOC, AND WIRELESS NETWORKS (WIOPT), 2021,
  • [39] Prototype Completion for Few-Shot Learning
    Zhang, Baoquan
    Li, Xutao
    Ye, Yunming
    Feng, Shanshan
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2023, 45 (10) : 12250 - 12268
  • [40] Few-Shot Learning With a Strong Teacher
    Ye, Han-Jia
    Ming, Lu
    Zhan, De-Chuan
    Chao, Wei-Lun
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2024, 46 (03) : 1425 - 1440