A Multi-Stage Transformer Network for Image Dehazing Based on Contrastive Learning

GAO Feng; JI Shengchang; GUO Jie; HOU Jie; OUYANG Chao; YANG Biao

doi:10.7652/xjtuxb202301019

您当前的位置：

首页 >

文章列表页 >

A Multi-Stage Transformer Network for Image Dehazing Based on Contrastive Learning

更新时间：2025-07-09

- A Multi-Stage Transformer Network for Image Dehazing Based on Contrastive Learning
- Vol. 57, Issue 1, Pages: 195-210(2023)
- 作者机构：
  
  1. 西安交通大学电气工程学院,西安,710049
  2. 哈尔滨工业大学(深圳)计算机科学与技术学院,广东,深圳,518055
  3. 哈尔滨工业大学(深圳)建筑学院, , ),广东,深圳,518055
- 作者简介：
- 基金信息：
- DOI：10.7652/xjtuxb202301019
  CLC： TP391
- Online First：10 January 2023，
  
  Published：2023
- 稿件说明：
移动端阅览
GAO Feng, JI Shengchang, GUO Jie, et al. A Multi-Stage Transformer Network for Image Dehazing Based on Contrastive Learning[J]. 2023, 57(1): 195-210.
DOI：

GAO Feng, JI Shengchang, GUO Jie, et al. A Multi-Stage Transformer Network for Image Dehazing Based on Contrastive Learning[J]. 2023, 57(1): 195-210. DOI： 10.7652/xjtuxb202301019.

摘要

为了解决现有图像去雾方法在图像局部去雾以及纹理细节恢复等方面始终不理想以及处理非均匀雾质始终不彻底的问题

提出了一种采用对比学习的多阶段自注意力模块(Transformer)的图像去雾MSTCNet方法。首先

利用信道级Transformer模块作为基本的特征提取模块

充分地捕获特征信道之间的长距离依赖关系; 其次

通过提出的多监督对比学习方法最大限度地挖掘正负样本信息

使去雾图像在投影后的隐空间中更靠近清晰图像

同时远离有雾图像; 最后

利用多阶段渐进式网络结构和可变形自注意力机制有效地整合图像局部细粒度特征和全局粗粒度信息。本文在2个合成数据集和3个真实数据集上对所提出的方法进行了大量的实验

结果表明:所提出的MSTCNet方法在5个数据集上的峰值信噪比(PSNR)分别提高了1.49、1.45、0.11、1.45和0.22 dB

在通用数据集与非数据集的测试中均超越已有的方法

在浓雾质、非均匀雾质以及均匀雾质的测试中均表现出最佳的去雾视觉效果

并达到最高的客观评价指标值。

Abstract

A multi-stage Transformer network for image dehazing based on contrastive learning is proposed to solve the problem that existing image dehazing methods fail to achieve the desired results in local image dehazing and detail restoration

and the non-homogeneous haze cannot be removed completely all the way. First

the channel-wise Transformer block is utilized as the primary feature extraction block to adequately capture the mutual long-range dependencies among channels. Second

the multi-modality supervised contrastive learning is introduced to maximize the capturing efficiency of information from the contrastive samples

so that the restored image is closer to the clear image in the embedding space while staying as far away from the hazy image as possible. Finally

a hierarchical multi-patch structure and deformable Transformer blocks are employed to effectively integrate the local and global structural information of the hazy image. Moreover

a large number of tests have been conducted on the proposed method by using two synthetic data sets and the three real data sets. The results show that the proposed MSTCNet achieves a higher peak signal-to-noise ratio(PSNR)gain of 1.49

1.45

0.11

1.45 and 0.22 dB on five datasets

respectively. It outperforms existing methods in both general and non-data sets

shows the best visual effect of dehazing in removing the dense

non-homogeneous and uniform haze

and achieves the highest objective evaluation index value.

关键词

Keywords

references

HASSAN H, MISHRA P, AHMAD M, et al. Effects of haze and dehazing on deep learning-based vision models [J/OL]. Applied Intelligence, 2022[2022-05-10].DOI:10.1007/s10489-022-03245-5.

NARASIMHAN S G, NAYAR S K. Vision and the atmosphere [J]. International Journal of Computer Vision, 2002, 48(3): 233-254.

HE Kaiming, SUN Jian, TANG Xiaoou. Single image haze removal using dark channel prior [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341-2353.

BERMAN D, TREIBITZ T, AVIDAN S. Non-local image dehazing [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ, USA: IEEE, 2016: 1674-1682.

黄文君, 李杰, 齐春. 低秩与字典表达分解的浓雾霾场景图像去雾算法 [J]. 西安交通大学学报, 2020, 54(4): 118-125.

HUANG Wenjun, LI Jie, QI Chun. A defogging algorithm for dense fog images via low-rank and dictionary expression decomposition [J]. Journal of Xi'an Jiaotong University, 2020, 54(4): 118-125.

高涛, 刘梦尼, 陈婷, 等. 结合暗亮通道先验的远近景融合去雾算法 [J]. 西安交通大学学报, 2021, 55(10): 78-86.

GAO Tao, LIU Mengni, CHEN Ting, et al. A far and near scene fusion defogging algorithm based on the prior of Dark-light channel [J]. Journal of Xi'an Jiaotong University, 2021, 55(10): 78-86.

杨燕, 陆鑫璇. 结合自适应亮度变换不等式估计透射率的图像去雾方法 [J]. 西安交通大学学报, 2021, 55(6): 69-76.

YANG Yan, LU Xinxuan. An image dehazing method combining adaptive brightness transformation inequality to estimate transmittance [J]. Journal of Xi'an Jiaotong University, 2021, 55(6): 69-76.

黄鹤, 胡凯益, 宋京, 等. 雾霾线求解透射率的二次优化方法 [J]. 西安交通大学学报, 2021, 55(8): 130-138.

HUANG He, HU Kaiyi, SONG Jing, et al. A twice optimization method for solving transmittance with haze-lines [J]. Journal of Xi'an Jiaotong University, 2021, 55(8): 130-138.

QU Yanyun, CHEN Yizi, HUANG Jingying, et al. Enhanced Pix2pix dehazing network [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ, USA: IEEE, 2019: 8152-8160.

QIN Xu, WANG Zhilin, BAI Yuanchao, et al. FFA-net: feature fusion attention network for single image dehazing [C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto, CA, USA: AAAI Press, 2020: 11908-11915.

ZHU Xinshan, LI Shuoshi, GAN Yongdong, et al. Multi-stream fusion network with generalized smooth L1 loss for single image dehazing [J]. IEEE Transactions on Image Processing, 2021, 30: 7620-7635.

YAN Jingke, WANG Xin, CAI Jingye, et al. Medical image segmentation model based on triple gate MultiLayer perceptron [J]. Scientific Reports, 2022, 12(1): 6103.

ZHAO Hengshuang, JIA Jiaya, KOLTUN V. Exploring self-attention for image recognition [C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ, USA: IEEE, 2020: 10073-10082.

ZAMIR S W, ARORA A, KHAN S, et al. Restormer: efficient transformer for high-resolution image restoration [J/OL]. arXiv, 2022[2022-01-10].DOI:10.48550/arXiv.2111.09881.

WU Haiyan, QU Yanyun, LIN Shaohui, et al. Contrastive learning for compact single image dehazing [C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ, USA: IEEE, 2021: 10546-10555.

ZAMIR S W, ARORA A, KHAN S, et al. Multi-stage progressive image restoration [C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ, USA: IEEE, 2021: 14816-14826.

ZHANG Hongguang, DAI Yuchao, LI Hongdong, et al. Deep stacked hierarchical multi-patch network for image deblurring [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ, USA: IEEE, 2019: 5971-5979.

TAO Xin, GAO Hongyun, SHEN Xiaoyong, et al. Scale-recurrent network for deep image deblurring [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2018: 8174-8182.

LIU Xiaohong, MA Yongrui, SHI Zhihao, et al. GridDehazeNet: attention-based multi-scale network for image dehazing [C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV). Piscataway, NJ, USA: IEEE, 2019: 7313-7322.

WANG Yanbo, LIN Shaohui, QU Yanyun, et al. Towards compact single image super-resolution via contrastive self-distillation [C]//Proceedings of the 30th International Joint Conference on Artificial Intelligence. Burlington,MA,USA:Morgan Kaufmann Publishers, 2021: 1122-1128.

WU Gang, JIANG Junjun, LIU Xianming, et al. A practical contrastive learning framework for single image super-resolution [J/OL]. arXiv, 2021[2022-02-01].DOI:10.48550/arXiv.2111.13924.

XIA Zhuofan, PAN Xuran, SONG Shiji, et al. Vision transformer with deformable attention [J/OL]. arXiv, 2022[2022-03-04].DOI:10.48550/arXiv.2201.00520.

CAI Bolun, XU Xiangmin, JIA Kui, et al. DehazeNet: an end-to-end system for single image haze removal [J]. IEEE Transactions on Image Processing, 2016, 25(11): 5187-5198.

REN Wenqi, LIU Si, ZHANG Hua, et al. Single image dehazing via multi-scale convolutional neural networks [C]//Computer Vision - ECCV 2016. Berlin, Germany: Springer International Publishing, 2016: 154-169.

HE Kaiming, FAN Haoqi, WU Yuxin, et al. Momentum contrast for unsupervised visual representation learning [C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ, USA: IEEE, 2020: 9726-9735.

JI Haobo, FENG Xin, PEI Wenjie, et al. U2-former: a nested u-shaped transformer for image restoration [J/OL]. arXiv, 2021[2022-01-20].DOI:10.48550/arXiv.2112.02279.

VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc., 2017: 6000-6010.

DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale [J/OL]. arXiv, 2020[2021-09-18].DOI:10.48550/arXiv.2010.11929.

LIU Ze, LIN Yutong, CAO Yue, et al. Swin transformer: hierarchical vision transformer using shifted windows [C]//2021 IEEE/CVF International Conference on Computer Vision(ICCV). Piscataway, NJ, USA: IEEE, 2021: 9992-10002.

HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2018: 7132-7141.

ANCUTI C O, ANCUTI C, SBERT M, et al. Dense-haze: a benchmark for image dehazing with dense-haze and haze-free images [C]//2019 IEEE International Conference on Image Processing(ICIP). Piscataway, NJ, USA: IEEE, 2019: 1014-1018.

ANCUTI C O, ANCUTI C, TIMOFTE R. NH-HAZE: an image dehazing benchmark with non-homogeneous hazy and haze-free images [C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW). Piscataway, NJ, USA: IEEE, 2020: 1798-1805.

ANCUTI C O, ANCUTI C, VASLUIANU F A, et al. NTIRE 2021 NonHomogeneous dehazing challenge report [C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW). Piscataway, NJ, USA: IEEE, 2021: 627-646.

LI Boyi, PENG Xiulian, WANG Zhangyang, et al. AOD-net: all-in-one dehazing network [C]//2017 IEEE International Conference on Computer Vision(ICCV). Piscataway, NJ, USA: IEEE, 2017: 4780-4788.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

An Image Dehazing Method Combining Adaptive Brightness Transformation Inequality to Estimate Transmittance

From Global to Local: A Dual-Attention Fusion Dehazing Network

Numerical Simulation of the Energy Conversion Process in Thermoacoustic Engines with Different Stack Materials

Influence of Structural Parameters on the Summer Operational Characteristics of a Three-Tower-in-One Indirect Air-Cooling System

Energy Efficiency Optimization of the Cryogenic Air Separation Process for Electronic-Grade Ultra Pure Nitrogen

Related Author

杨燕

陆鑫璇

YANG Aiwei

WANG Huake

HOU Xingsong

Related Institution

School of Electronic and Information Engineering, Lanzhou Jiaotong University

(Faculty of Electronic and Information Engineering, Xi'an Jiaotong University,,)

⁰