Prompt-Based Test-Time Real Image Dehazing: A Novel Pipeline

被引:0
|
作者
Chen, Zixuan [1 ]
He, Zewei [1 ,2 ]
Lu, Ziqian [1 ]
Sun, Xuecheng [1 ]
Lu, Zhe-Ming [1 ,2 ]
机构
[1] Zhejiang Univ, Sch Aeronaut & Astronaut, Hangzhou, Peoples R China
[2] Huanjiang Lab, Hangzhou, Peoples R China
来源
基金
中国国家自然科学基金; 中国博士后科学基金;
关键词
Image dehazing; Visual prompt; Feature adaptation;
D O I
10.1007/978-3-031-73116-7_25
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Existing methods attempt to improve models' generalization ability on real-world hazy images by exploring well-designed training schemes (e.g., CycleGAN, prior loss). However, most of them need very complicated training procedures to achieve satisfactory results. For the first time, we present a novel pipeline called Prompt-based Test-Time Dehazing (PTTD) to help generate visually pleasing results of real-captured hazy images during the inference phase. We experimentally observe that given a dehazing model trained on synthetic data, fine-tuning the statistics (i.e., mean and standard deviation) of encoding features is able to narrow the domain gap, boosting the performance of real image dehazing. Accordingly, we first apply a prompt generation module (PGM) to generate a visual prompt, which is the reference of appropriate statistical perturbations for mean and standard deviation. Then, we employ a feature adaptation module (FAM) into the existing dehazing models for adjusting the original statistics with the guidance of the generated prompt. PTTD is model-agnostic and can be equipped with various state-of-the-art dehazing models trained on synthetic hazy-clean pairs to tackle the real image dehazing task. Extensive experimental results demonstrate that our PTTD is effective, achieving superior performance against state-of-the-art dehazing methods in real-world scenarios. The code is available at https://github.com/cecret3350/PTTD-Dehazing.
引用
收藏
页码:432 / 449
页数:18
相关论文
共 50 条
  • [1] Towards Multi-domain Single Image Dehazing via Test-time Training
    Liu, Huan
    Wu, Zijun
    Li, Liangyan
    Salehkalaibar, Sadaf
    Chen, Jun
    Wang, Keyan
    2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, : 5821 - 5830
  • [2] Prompt-Based Learning for Unpaired Image Captioning
    Zhu, Peipei
    Wang, Xiao
    Zhu, Lin
    Sun, Zhenglong
    Zheng, Wei-Shi
    Wang, Yaowei
    Chen, Changwen
    IEEE TRANSACTIONS ON MULTIMEDIA, 2024, 26 : 379 - 393
  • [3] Test-Time Personalization with Meta Prompt for Gaze Estimation
    Liu, Huan
    Qi, Julia
    Li, Zhenhao
    Hassanpour, Mohammad
    Wang, Yang
    Plataniotis, Konstantinos
    Yu, Yuanhao
    THIRTY-EIGHTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 38 NO 4, 2024, : 3621 - 3629
  • [4] VPA: Fully Test-Time Visual Prompt Adaptation
    Sun, Jiachen
    Ibrahim, Mark
    Hall, Melissa
    Evtimov, Ivan
    Mao, Z. Morley
    Ferrer, Cristian Canton
    Hazirbas, Caner
    PROCEEDINGS OF THE 31ST ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2023, 2023, : 5796 - 5806
  • [5] A Prompt-Based Hierarchical Pipeline for Cross-Domain Slot Filling
    Wei, Xiao
    Li, Yuhang
    Si, Yuke
    Wang, Longbiao
    Wang, Xiaobao
    Dang, Jianwu
    IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING, 2024, 32 : 3061 - 3075
  • [6] Fully Test-Time Adaptation for Image Segmentation
    Hu, Minhao
    Song, Tao
    Gu, Yujun
    Luo, Xiangde
    Chen, Jieneng
    Chen, Yinan
    Zhang, Ya
    Zhang, Shaoting
    MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION - MICCAI 2021, PT III, 2021, 12903 : 251 - 260
  • [7] Test-Time Adaptation for Deformable Image Registration
    Sang, Y.
    McNitt-Gray, M.
    Yang, Y.
    Cao, M.
    Low, D.
    Ruan, D.
    MEDICAL PHYSICS, 2022, 49 (06) : E458 - E459
  • [8] Robust Test-Time Adaptation for Zero-Shot Prompt Tuning
    Zhang, Ding-Chu
    Zhou, Zhi
    Li, Yu-Feng
    THIRTY-EIGHTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 38 NO 15, 2024, : 16714 - 16722
  • [9] Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuning
    Feng, Chun-Mei
    Yu, Kai
    Liu, Yong
    Khan, Salman
    Zuo, Wangmeng
    2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION, ICCV, 2023, : 2704 - 2714
  • [10] FATH: Authentication-based Test-time Defense against Indirect Prompt Injection Attacks
    Wang, Jiongxiao
    Wu, Fangzhou
    Li, Wendi
    Pan, Jinsheng
    Suh, Edward
    Mao, Z. Morley
    Chen, Muhao
    Xiao, Chaowei
    arXiv,