An Efficient Infrared Image Colorization Method Based on Dual-Branch Feature Interaction Fusion

CHEN Yu; ZHAN Weida; JIANG Yichun; ZHU Depeng; HAN Deng

doi:10.7652/xjtuxb202508020

您当前的位置：

首页 >

文章列表页 >

An Efficient Infrared Image Colorization Method Based on Dual-Branch Feature Interaction Fusion

更新时间：2025-07-30

- An Efficient Infrared Image Colorization Method Based on Dual-Branch Feature Interaction Fusion
- Journal of Xi’an Jiaotong University Vol. 59, Issue 8, Pages: 211-222(2025)
- 作者机构：
  
  1.长春理工大学电子信息工程学院， 130022，长春
  2.吉林省知行物联网研究院有限公司， 130117，长春
- 作者简介：
- 基金信息：
- DOI：10.7652/xjtuxb202508020
  CLC： TN911.73;TP391
- Received：21 February 2025，
  
  Online First：19 March 2025，
  
  Published：10 August 2025
- 稿件说明：
移动端阅览
CHEN Yu, ZHAN Weida, JIANG Yichun, et al. An Efficient Infrared Image Colorization Method Based on Dual-Branch Feature Interaction Fusion[J]. Journal of Xi’an Jiaotong University, 2025, 59(8): 211-222.
DOI：

CHEN Yu, ZHAN Weida, JIANG Yichun, et al. An Efficient Infrared Image Colorization Method Based on Dual-Branch Feature Interaction Fusion[J]. Journal of Xi’an Jiaotong University, 2025, 59(8): 211-222. DOI： 10.7652/xjtuxb202508020.

摘要

针对现有的红外图像彩色化方法在全局特征捕获和计算复杂度方面存在显著局限性的问题，提出了一种双分支特征交互融合的高效红外图像彩色化方法。设计双分支编码器，通过局部特征提取分支获取局部空间上下文信息，确保细粒度特征的捕获，并通过全局特征提取分支获取全局特征，满足对长程依赖的需求。设计交互融合模块，对两个分支提取到的特征进行有效整合，显著增强了模型的整体性能。在解码器部分提出上下文聚合模块，进一步优化多尺度语义特征的聚合能力，改善了彩色化结果的边缘清晰度和细节表现力。在KAIST和FLIR数据集上进行广泛实验验证，结果表明：与现有方法相比，所提方法在两个数据集上均具有更高的彩色化质量，峰值信噪比分别达到28.645、30.459 dB，结构相似度达到0.507、0.725，均优于对比方法，且有效性和先进性也得到了验证。研究结果可为提升红外图像的可读性与可解释性以及提高夜视与恶劣环境下的观测能力提供参考。

Abstract

To address the significant limitations of existing infrared image colorization methods in global feature capture and computational complexity

an efficient infrared image colorization method based on dual-branch feature interaction fusion is proposed. A dual-branch encoder is designed

where the local feature extraction branch captures local spatial context information to ensure fine-grained feature acquisition

while the global feature extraction branch obtains global features to meet long-range dependency requirements. An interaction fusion module is developed to effectively integrate features extracted from both branches

significantly enhancing the model's overall performance. In the decoder part

a context aggregation module is proposed to further optimize multi-scale semantic feature aggregation

improving edge clarity and detail representation in the colorization results. Extensive experimental validation on the KAIST and FLIR datasets demonstrates that compared to existing methods

the proposed approach achieves superior colorization quality on both datasets

with peak signal-to-noise ratios reaching 28.645 dB and 30.459 dB

and structural similarity scores of 0.507 and 0.725

respectively

outperforming all comparative methods. The effectiveness and advancement of the method are thus verified. The research findings provide valuable references for enhancing the readability and interpretability of infrared images

as well as improving observation capabilities in night vision and harsh environments.

关键词

Keywords

references

林再平 , 罗伊杭 , 李博扬 , 等 . 基于梯度可感知通道注意力模块的红外小目标检测前去噪网络 [J ] . 红外与毫米波学报 , 2024 , 43 ( 2 ): 254 - 260 .

LIN Zaiping , LUO Yihang , LI Boyang , et al . Gradient-aware channel attention network for infrared small target image denoising before detection [J ] . Journal of Infrared and Millimeter Waves , 2024 , 43 ( 2 ): 254 - 260 .

CHEN Yu , ZHAN Weida , JIANG Yichun , et al . A feature refinement and adaptive generative adversarial network for thermal infrared image colorization [J ] . Neural Networks , 2024 , 173 : 106184 .

CHEN Yu , ZHAN Weida , JIANG Yichun , et al . Contrastive learning with feature fusion for unpaired thermal infrared image colorization [J ] . Optics and Lasers in Engineering , 2023 , 170 : 107745 .

RONNEBERGER O , FISCHER P , BROX T . U-Net: convolutional networks for biomedical image segmentation [C ] // Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015 . Cham, Germany : Springer International Publishing , 2015 : 234 - 241 .

ASWATHA S M , MALLADI S P K , MUKHERJEE J . An encoder-decoder based deep architecture for visible to near infrared image transformation [C ] // Proceedings of the Twelfth Indian Conference on Computer Vision, Graphics and Image Processing . New York, USA : ACM , 2021 : 29 .

ISOLA P , ZHU Junyan , ZHOU Tinghui , et al . Image-to-image translation with conditional adversarial networks [C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway, NJ, USA : IEEE , 2017 : 5967 - 5976 .

GOODFELLOW I J , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial nets [C ] // Proceedings of the 27th International Conference on Neural Information Processing Systems . Cambridge, MA, USA : MIT Press , 2014 : 2672 - 2680 .

KUANG Xiaodong , ZHU Jianfei , SUI Xiubao , et al . Thermal infrared colorization via conditional generative adversarial network [J ] . Infrared Physics & Technology , 2020 , 107 : 103338 .

LIAO Hangying , JIANG Qian , JIN Xin , et al . MUGAN: thermal infrared image colorization using mixed-skipping UNet and generative adversarial network [J ] . IEEE Transactions on Intelligent Vehicles , 2023 , 8 ( 4 ): 2954 - 2969 .

朴燕 , 康继元 . RAUGAN：基于循环生成对抗网络的红外图像彩色化方法 [J/OL ] . 吉林大学学报(工学版) . ( 2024-04-28 ) [ 2024-06-20 ] . https://doi.org/10.13229/j.cnki.jdxbgxb.20240012 https://doi.org/10.13229/j.cnki.jdxbgxb.20240012 .

PIAO Yan , KANG Jiyuan . RAUGAN: infrared image colorization method based on cycle generative adversarial networks [J/OL ] . Journal of Jilin University (Engineering and Technology Edition) . ( 2024-04-28 ) [ 2024-06-20 ] . https://doi.org/10.13229/j.cnki.jdxbgxb.20240012 https://doi.org/10.13229/j.cnki.jdxbgxb.20240012 .

赵伟强 , 王洋 , 祝勇 , 等 . 基于颜色预测和语义感知双分支结构的近红外图像彩色化算法 [J ] . 光学技术 , 2023 , 49 ( 3 ): 371 - 378 .

ZHAO Weiqiang , WANG Yang , ZHU Yong , et al . Colorization algorithm for near infrared images based on dual branching structure of color prediction and semantic perception [J ] . Optical Technique , 2023 , 49 ( 3 ): 371 - 378 .

DOSOVITSKIY A , BEYER L , KOLESNIKOV A , et al . An image is worth 16×16 words: transformers for image recognition at scale [C ] // Proceedings of the International Conference on Learning Representations . New York, USA : ICLR , 2021 : 1 - 22 .

TORBUNOV D , HUANG Yi , YU Haiwang , et al . UVCGAN: UNet vision transformer cycle-consistent GAN for unpaired image-to-image translation [C ] // 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) . Piscataway, NJ, USA : IEEE , 2023 : 702 - 712 .

HE Youwei , JIN Xin , JIANG Qian , et al . LKAT-GAN: a GAN for thermal infrared image colorization based on large kernel and attentionUNet-transformer [J ] . IEEE Transactions on Consumer Electronics , 2023 , 69 ( 3 ): 478 - 489 .

GU A , DAO T . Mamba: linear-time sequence modeling with selective state spaces [EB/OL ] . ( 2024-05-31 ) [ 2024-05-31 ] . https://arxiv.org/abs/2312.00752 https://arxiv.org/abs/2312.00752 .

LIU Yue , TIAN Yunjie , ZHAO Yuzhong , et al . Vmamba: visual state space model [C ] // Proceedings of the Advances in Neural Information Processing Systems . San Diego : NIPS Press , 2024 : 103031 - 103063 .

CHEN Keyan , CHEN Bowen , LIU Chenyang , et al . RSMamba: remote sensing image classification with state space model [J ] . IEEE Geoscience and Remote Sensing Letters , 2024 , 21 : 1 - 5 .

GUO Hang , LI Jinmin , DAI Tao , et al . MambaIR: a simple baseline for image restoration with state-space model [C ] // Computer Vision-ECCV 2024 . Cham : Springer Nature Switzerland , 2025 : 222 - 241 .

SUN Shangquan , REN Wenqi , ZHOU Juxiang , et al . A hybrid transformer-mamba network for single image deraining [EB/OL ] . ( 2024-08-31 ) [ 2024-08-31 ] . https://arxiv.org/abs/2409.00410 https://arxiv.org/abs/2409.00410 .

ZHAI Huiyu , JIN Guang , YANG Xingxing , et al . ColorMamba: towards high-quality NIR-to-RGB spectral translation with mamba [C ] // Proceedings of the 16th Asian Conference on Machine Learning . Chia Laguna Resort, Sardinia, Italy : PMLR , 2025 : 765 - 780 .

HAN Dongchen , YE Tianzhu , HAN Yizeng , et al . Agent attention: on the integration of softmax and linear attention [C ] // Computer Vision-ECCV 2024 . Cham : Springer Nature Switzerland , 2025 : 124 - 140 .

CHEN Yu , ZHAN Weida , JIANG Yichun , et al . DDGAN: dense residual module and dual-stream attention-guided generative adversarial network for colorizing near-infrared images [J ] . Infrared Physics & Technology , 2023 , 133 : 104822 .

HWANG S , PARK J , KIM N , et al . Multispectral pedestrian detection: benchmark dataset and baseline [C ] // 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway, NJ, USA : IEEE , 2015 : 1037 - 1045 .

CHEN Yu , ZHAN Weida , JIANG Yichun , et al . Exploring efficient and effective generative adversarial network for thermal infrared image colorization [J ] . Complex & Intelligent Systems , 2023 , 9 ( 6 ): 7015 - 7036 .

RUAN Jiacheng , LI Jincheng , XIANG Suncheng . VM-UNet: vision mamba uNet for medical image segmentation [EB/OL ] . ( 2024-11-08 ) [ 2024-11-08 ] . https://arxiv.org/abs/2402.02491 https://arxiv.org/abs/2402.02491 .

吴洪伍 , 盖绍彦 , 达飞鹏 . 采用感受野优化与渐进特征融合的图像超分辨率算法 [J ] . 西安交通大学学报 , 2025 , 59 ( 1 ): 136 - 147 .

WU Hongwu , GE Shaoyan , DA Feipeng . Image super resolution algorithm based on receptive field optimization and progressive feature fusion [J ] . Journal of Xi'an Jiaotong University , 2025 , 59 ( 1 ): 136 - 147 .

YADAV N K , SINGH S K , DUBEY S R . MobileAR-GAN: MobileNet-based efficient attentive recurrent generative adversarial network for infrared-to-visual transformations [J ] . IEEE Transactions on Instrumentation and Measurement , 2022 , 71 : 1 - 9 .

HUANG Junjian , REN Hao , LIU Shulin , et al . Real-time attentive dilated U-Net for extremely dark image enhancement [J ] . ACM Transactions on Multimedia Computing, Communications, and Applications , 2024 , 20 ( 8 ): 231 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Numerical Simulation of the Energy Conversion Process in Thermoacoustic Engines with Different Stack Materials

Mechanism and Optimal Control Strategy for Hysteresis in Rapid Dehumidification Response of Indirect Thermal Systems in Electric Vehicles

A Fast Calculation Method for Thermal Flow Fields in Transformer Windings Based on PI-DeepONet

Knowledge Graph Representation Learning Driven by Semantic Discrimination and Subgraph Integration

Turbulent Kinetic Energy Dissipation Mechanism of Supercritical Carbon Dioxide Leakage Flow in a Scallop Bionic Seal Cavity

Related Author

No data

Related Institution

No data

⁰