SelFlow: Self-Supervised Learning of Optical Flow

被引:216
|
作者
Liu, Pengpeng [1 ,2 ]
Lyu, Michael [1 ]
King, Irwin [1 ]
Xu, Jia [2 ]
机构
[1] Chinese Univ Hong Kong, Hong Kong, Peoples R China
[2] Tencent AI Lab, Bellevue, WA USA
关键词
D O I
10.1109/CVPR.2019.00470
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We present a self-supervised learning approach for optical flow Our method distills reliable flow estimations from non-occluded pixels, and uses these predictions as ground truth to learn optical flow for hallucinated occlusions. We further design a simple CNN to utilize temporal information from multiple frames for better flow estimation. These two principles lead to an approach that yields the best performance for unsupervised optical flow learning on the challenging benchmarks including MPI Sintel, KITTI 2012 and 2015. More notably, our self-supervised pre-trained model provides an excellent initialization for supervised fine-tuning. Our fine-tuned models achieve stateof-the-art results on all three datasets. At the time of writing, we achieve EPE=4.26 on the Sintel benchmark, outperforming all submitted methods.
引用
收藏
页码:4566 / 4575
页数:10
相关论文
共 50 条
  • [1] Optical flow for self-supervised learning of obstacle appearance
    Ho, H. W.
    De Wagter, C.
    Remes, B. D. W.
    de Croon, G. C. H. E.
    2015 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS), 2015, : 3098 - 3104
  • [2] Learning by Distillation: A Self-Supervised Learning Framework for Optical Flow Estimation
    Liu, Pengpeng
    Lyu, Michael R.
    King, Irwin
    Xu, Jia
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2022, 44 (09) : 5026 - 5041
  • [3] Cross Pixel Optical-Flow Similarity for Self-supervised Learning
    Mahendran, Aravindh
    Thewlis, James
    Vedaldi, Andrea
    COMPUTER VISION - ACCV 2018, PT V, 2019, 11365 : 99 - 116
  • [4] Flow2Stereo: Effective Self-Supervised Learning of Optical Flow and Stereo Matching
    Liu, Pengpeng
    King, Irwin
    Lyu, Michael R.
    Xu, Jia
    2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, : 6647 - 6656
  • [5] Self-Supervised Optical Flow Estimation by Projective Bootstrap
    Alletto, Stefano
    Abati, Davide
    Calderara, Simone
    Cucchiara, Rita
    Rigazio, Luca
    IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2019, 20 (09) : 3294 - 3302
  • [6] Self-supervised Segmentation by Grouping Optical-Flow
    Mahendran, Aravindh
    Thewlis, James
    Vedaldi, Andrea
    COMPUTER VISION - ECCV 2018 WORKSHOPS, PT V, 2019, 11133 : 528 - 534
  • [7] Contrastive Self-Supervised Learning for Optical Music Recognition
    Penarrubia, Carlos
    Valero-Mas, Jose J.
    Calvo-Zaragoza, Jorge
    DOCUMENT ANALYSIS SYSTEMS, DAS 2024, 2024, 14994 : 312 - 326
  • [8] Reconfigurable Optical Datacom Networks by Self-supervised Learning
    Liu, Che-Yu
    Chen, Xiaoliang
    Proietti, Roberto
    Li, Zhaohui
    Ben Yoo, S. J.
    PROCEEDINGS OF THE 2021 ACM SIGCOMM 2021 WORKSHOP ON OPTICAL SYSTEMS (OPTSYS '21), 2021, : 23 - 27
  • [9] Self-Supervised Representation Learning from Flow Equivariance
    Xiong, Yuwen
    Ren, Mengye
    Zeng, Wenyuan
    Urtasun, Raquel
    2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, : 10171 - 10180
  • [10] Self-supervised learning for effective denoising of flow fields
    Yu, Linqi
    Yousif, Mustafa Z.
    Zhou, Dan
    Zhang, Meng
    Lee, Jung Sub
    Lim, Hee-Chang
    PHYSICS OF FLUIDS, 2024, 36 (10)