Detecting Eating and Social Presence with All DayWearable RGB-T

被引:1
|
作者
Shahi, Soroush [1 ]
Sen, Sougata [2 ]
Pedram, Mahdi [1 ]
Alharbi, Rawan [1 ]
Gao, Yang [1 ]
Katsaggelos, Aggelos K. [1 ]
Hester, Josiah [3 ]
Alshurafa, Nabil [1 ]
机构
[1] Northwestern Univ, Evanston, IL 60208 USA
[2] Birla Inst Technol & Sci, Pilani, Goa, India
[3] Georgia Inst Technol, Atlanta, GA 30332 USA
来源
2023 IEEE/ACM CONFERENCE ON CONNECTED HEALTH: APPLICATIONS, SYSTEMS AND ENGINEERING TECHNOLOGIES, CHASE | 2023年
关键词
human activity recognition; wearable camera; deep learning; FOOD-INTAKE; IMPACT; REAL;
D O I
10.1145/3580252.3586974
中图分类号
TP3 [计算技术、计算机技术];
学科分类号
0812 ;
摘要
Social presence has been known to impact eating behavior among people with obesity; however, the dual study of eating behavior and social presence in real-world settings is challenging due to the inability to reliably confirm the co-occurrence of these important factors. High-resolution video cameras can detect timing while providing visual confirmation of behavior; however, their potential to capture all-day behavior is limited by short battery lifetime and lack of autonomy in detection. Low-resolution infrared (IR) sensors have shown promise in automating human behavior detection; however, it is unknown if IR sensors contribute to behavior detection when combined with RGB cameras. To address these challenges, we designed and deployed a low-power, and low-resolution RGB video camera, in conjunction with a low-resolution IR sensor, to test a learned model's ability to detect eating and social presence. We evaluated our system in the wild with 10 participants with obesity; our models displayed slight improvement when detecting eating (5%) and significant improvement when detecting social presence (44%) compared with using a video-only approach. We analyzed device failure scenarios and their implications for future wearable camera design and machine learning pipelines. Lastly, we provide guidance for future studies using low-cost RGB and IR sensors to validate human behavior with context.
引用
收藏
页码:68 / 79
页数:12
相关论文
共 50 条
  • [1] RGB-T目标跟踪综述
    丁正彤
    徐磊
    张研
    李飘扬
    李阳阳
    罗斌
    涂铮铮
    南京信息工程大学学报(自然科学版), 2019, 11 (06) : 690 - 697
  • [2] MFENet: Multitype fusion and enhancement network for detecting salient objects in RGB-T images
    Wu J.
    Zhou W.
    Qian X.
    Lei J.
    Yu L.
    Luo T.
    Digital Signal Processing: A Review Journal, 2023, 133
  • [3] Channel Exchanging for RGB-T Tracking
    Zhao, Long
    Zhu, Meng
    Ren, Honge
    Xue, Lingjixuan
    SENSORS, 2021, 21 (17)
  • [4] RGB-T object tracking: Benchmark and baseline
    Li, Chenglong
    Liang, Xinyan
    Lu, Yijuan
    Zhao, Nan
    Tang, Jin
    PATTERN RECOGNITION, 2019, 96
  • [5] RGB-T tracking with frequency hybrid awareness
    Lei, Lei
    Li, Xianxian
    IMAGE AND VISION COMPUTING, 2024, 152
  • [6] Hierarchical Decoding Network Based on Swin Transformer for Detecting Salient Objects in RGB-T Images
    Sun, Fan
    Zhou, Wujie
    Ye, Lv
    Yu, Lu
    IEEE SIGNAL PROCESSING LETTERS, 2022, 29 : 1714 - 1718
  • [7] Toward Modalities Correlation for RGB-T Tracking
    Hu, Xiantao
    Zhong, Bineng
    Liang, Qihua
    Zhang, Shengping
    Li, Ning
    Li, Xianxian
    IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2024, 34 (10) : 9102 - 9111
  • [8] RGB-T Object Detection With Failure Scenarios
    Wang, Qingwang
    Sun, Yuxuan
    Chi, Yongke
    Shen, Tao
    IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 2025, 18 : 3000 - 3010
  • [9] Saliency Prototype for RGB-D and RGB-T Salient Object Detection
    Zhang, Zihao
    Wang, Jie
    Han, Yahong
    PROCEEDINGS OF THE 31ST ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2023, 2023, : 3696 - 3705
  • [10] Resolving semantic conflicts in RGB-T semantic segmentation
    Zhao, Shenlu
    Jin, Ziniu
    Jiao, Qiang
    Zhang, Qiang
    Han, Jungong
    PATTERN RECOGNITION, 2025, 162