Automatic Recognition and Augmentation of Attended Objects in Real-time using Eye Tracking and a Head-mounted Display

被引：11
|
作者：
Barz, Michael [1 ]
Kapp, Sebastian [2 ]
Kuhn, Jochen [2 ]
Sonntag, Daniel [1 ]
机构：
[1] Oldenburg Univ, German Res Ctr Artificial Intelligence DFKI, Oldenburg, Germany
[2] Tech Univ Kaiserslautern, Kaiserslautern, Rlp, Germany
来源：
ACM SYMPOSIUM ON EYE TRACKING RESEARCH AND APPLICATIONS (ETRA 2021) | 2021年
关键词：
eye tracking; augmented reality; visual attention; cognition-aware computing; computer vision;
D O I：
10.1145/3450341.3458766
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
Scanning and processing visual stimuli in a scene is essential for the human brain to make situation-aware decisions. Adding the ability to observe the scanning behavior and scene processing to intelligent mobile user interfaces can facilitate a new class of cognition-aware user interfaces. As a first step in this direction, we implement an augmented reality (AR) system that classifies objects at the user's point of regard, detects visual attention to them, and augments the real objects with virtual labels that stick to the objects in real-time. We use a head-mounted AR device (Microsoft HoloLens 2) with integrated eye tracking capabilities and a front-facing camera for implementing our prototype.
引用
收藏
页数：4
相关论文
共 50 条
  • [1] Real-time Gaze Tracking with Head-eye Coordination for Head-mounted Displays
    Chen, Lingling
    Li, Yingxi
    Bai, Xiaowei
    Wang, Xiaodong
    Hu, Yongqiang
    Song, Mingwu
    Xie, Liang
    Yan, Ye
    Yin, Erwei
    2022 IEEE INTERNATIONAL SYMPOSIUM ON MIXED AND AUGMENTED REALITY (ISMAR 2022), 2022, : 82 - 91
  • [2] Real-Time Recognition of In-Place Body Actions and Head Gestures using Onlya Head-Mounted Display
    Zhao, Jingbo
    Shao, Mingjun
    Wang, Yaojun
    Xu, Ruolin
    2023 IEEE CONFERENCE VIRTUAL REALITY AND 3D USER INTERFACES, VR, 2023, : 105 - 114
  • [3] Real-Time Step Detection Using the Integrated Sensors of a Head-Mounted Display
    Caserman, Polona
    Krabbe, Patrick
    Wojtusch, Janis
    von Stryk, Oskar
    2016 IEEE INTERNATIONAL CONFERENCE ON SYSTEMS, MAN, AND CYBERNETICS (SMC), 2016, : 3510 - 3515
  • [4] EyeRecToo: Open-source Software for Real-time Pervasive Head-mounted Eye Tracking
    Santini, Thiago
    Fuhl, Wolfgang
    Geisler, David
    Kasneci, Enkelejda
    PROCEEDINGS OF THE 12TH INTERNATIONAL JOINT CONFERENCE ON COMPUTER VISION, IMAGING AND COMPUTER GRAPHICS THEORY AND APPLICATIONS (VISIGRAPP 2017), VOL 6, 2017, : 96 - 101
  • [5] Head-mounted eye movement tracking display and its image display method
    AIST-MITI, Tokyo, Japan
    Systems and Computers in Japan, 1997, 28 (07): : 89 - 99
  • [6] Individualized foveated rendering with eye-tracking head-mounted display
    Kim, Jihwan
    Kim, Jejoong
    Jung, Myeongul
    Kwon, Taesoo
    Kim, Kwanguk Kenny
    VIRTUAL REALITY, 2024, 28 (01)
  • [7] Individualized foveated rendering with eye-tracking head-mounted display
    Jihwan Kim
    Jejoong Kim
    Myeongul Jung
    Taesoo Kwon
    Kwanguk Kenny Kim
    Virtual Reality, 2024, 28
  • [8] Real-Time Head Gesture Recognition on Head-Mounted Displays using Cascaded Hidden Markov Models
    Zhao, Jingbo
    Allison, Robert S.
    2017 IEEE INTERNATIONAL CONFERENCE ON SYSTEMS, MAN, AND CYBERNETICS (SMC), 2017, : 2361 - 2366
  • [9] Strategies for enhancing automatic fixation detection in head-mounted eye tracking
    Drews, Michael
    Dierkes, Kai
    BEHAVIOR RESEARCH METHODS, 2024, 56 (06) : 6276 - 6298
  • [10] An Immersive Telepresence System using a Real-Time Omnidirectional Camera and a Virtual Reality Head-Mounted Display
    Gaemperle, Luis
    Seyid, Kerem
    Popovic, Vladan
    Leblebici, Yusuf
    2014 IEEE INTERNATIONAL SYMPOSIUM ON MULTIMEDIA (ISM), 2014, : 175 - 178
12345