Spatio-Temporal Turbulence Mitigation: A Translational Perspective

被引:1
|
作者
Zhang, Xingguang [1 ]
Chimitt, Nicholas [1 ]
Chi, Yiheng [1 ]
Mao, Zhiyuan [2 ]
Chan, Stanley H. [1 ]
机构
[1] Purdue Univ, Sch Elect & Comp Engn, W Lafayette, IN 47907 USA
[2] Samsung Res Amer, Mountain View, CA USA
来源
2024 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2024 | 2024年
基金
美国国家科学基金会;
关键词
ATMOSPHERIC-TURBULENCE; VIDEO STABILIZATION; IMAGE;
D O I
10.1109/CVPR52733.2024.00279
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Recovering images distorted by atmospheric turbulence is a challenging inverse problem due to the stochastic nature of turbulence. Although numerous turbulence mitigation (TM) algorithms have been proposed, their efficiency and generalization to real-world dynamic scenarios remain severely limited. Building upon the intuitions of classical TM algorithms, we present the Deep Atmospheric TUrbulence Mitigation network (DATUM). DATUM aims to overcome major challenges when transitioning from classical to deep learning approaches. By carefully integrating the merits of classical multi-frame TM methods into a deep network structure, we demonstrate that DATUM can efficiently perform long-range temporal aggregation using a recurrent fashion, while deformable attention and temporal-channel attention seamlessly facilitate pixel registration and lucky imaging. With additional supervision, tilt and blur degradation can be jointly mitigated. These inductive biases empower DATUM to significantly outperform existing methods while delivering a tenfold increase in processing speed. A large-scale training dataset, ATSyn, is presented as a co-invention to enable the generalization to real turbulence. Our code and datasets are available at https://xg416.github.io/DATUM
引用
收藏
页码:2889 / 2899
页数:11
相关论文
共 50 条
  • [1] Kinetic model of the spatio-temporal turbulence
    Aristov, Vladimir
    Ilyin, Oleg
    PHYSICS LETTERS A, 2010, 374 (43) : 4381 - 4384
  • [2] On the spatio-temporal behavior of magnetohydrodynamic turbulence in a magnetized plasma
    Lugones, R.
    Dmitruk, P.
    Mininni, P. D.
    Wan, M.
    Matthaeus, W. H.
    PHYSICS OF PLASMAS, 2016, 23 (11)
  • [3] Lead and delinquency rates; A spatio-temporal perspective
    Mayer, Duncan J.
    SOCIAL SCIENCE & MEDICINE, 2024, 341
  • [4] Simulation of spatio-temporal turbulence on the basis of the discrete kinetic system
    Aristov, V. V.
    Ilyin, O. V.
    ICCS 2010 - INTERNATIONAL CONFERENCE ON COMPUTATIONAL SCIENCE, PROCEEDINGS, 2010, 1 (01): : 725 - 734
  • [5] Spatio-temporal dynamics of turbulence trapped in geodesic acoustic modes
    Sasaki, M.
    Kobayashi, T.
    Itoh, K.
    Kasuya, N.
    Kosuga, Y.
    Fujisawa, A.
    Itoh, S. -I.
    PHYSICS OF PLASMAS, 2018, 25 (01)
  • [6] MISOLFA: a generalized monitor for daytime spatio-temporal turbulence characterization
    Ikhlef, R.
    Corbard, T.
    Morand, F.
    Renaud, C.
    Fodil, M.
    Ziad, A.
    Borgnino, J.
    Meftah, M.
    Assus, P.
    Chauvineau, B.
    Hauchecorne, A.
    Lesueur, P.
    Poiet, G.
    Ubaldi, F.
    Hamadouche, M.
    Abdelatif, T.
    MONTHLY NOTICES OF THE ROYAL ASTRONOMICAL SOCIETY, 2016, 458 (01) : 517 - 530
  • [7] Perspective on the spatio-temporal spread of epidemics in metapopulation networks
    Wang, Jianbo
    Du, Zhanwei
    Li, Hui-Jia
    Wang, Lei
    Xu, Xiao-Ke
    Wang, Zhen
    Wang, Lin
    Li, Xiang
    EPL, 2023, 144 (01)
  • [8] Transportation Service Redundancy From a Spatio-Temporal Perspective
    Haddad, Hedi
    Bouyahia, Zied
    Jabeur, Nafaa
    IEEE INTELLIGENT TRANSPORTATION SYSTEMS MAGAZINE, 2019, 11 (04) : 157 - 166
  • [9] A Spatio-Temporal Perspective to Knowledge Management in the Construction Sector
    Mirarchi, Claudio
    IN BO-RICERCHE E PROGETTI PER IL TERRITORIO LA CITTA E L ARCHITETTURA, 2018, 9 (13): : 20 - 27
  • [10] Spatio-Temporal Prediction of Suspect Location by Spatio-Temporal Semantics
    Duan L.
    Hu T.
    Zhu X.
    Ye X.
    Wang S.
    Wuhan Daxue Xuebao (Xinxi Kexue Ban)/Geomatics and Information Science of Wuhan University, 2019, 44 (05): : 765 - 770