Infrared-Visible Heterogenous Image Matching Based on Intra-Class Transfer Learning

doi:10.7652/xjtuxb202001007

您当前的位置：

首页 >

文章列表页 >

Infrared-Visible Heterogenous Image Matching Based on Intra-Class Transfer Learning

更新时间：2025-07-09

- Infrared-Visible Heterogenous Image Matching Based on Intra-Class Transfer Learning
- Vol. 54, Issue 1, Pages: 49-55(2020)
- 作者机构：
  
  西安微电子技术研究所,西安,710065
- 作者简介：
- 基金信息：
- DOI：10.7652/xjtuxb202001007
  CLC： TP391
- Online First：10 January 2020，
  
  Published：2020
- 稿件说明：
移动端阅览
Infrared-Visible Heterogenous Image Matching Based on Intra-Class Transfer Learning[J]. 2020, 54(1): 49-55.
DOI：

Infrared-Visible Heterogenous Image Matching Based on Intra-Class Transfer Learning[J]. 2020, 54(1): 49-55. DOI： 10.7652/xjtuxb202001007.

摘要

为解决异源图像匹配中样本量过少和成像原理不同导致成像差异的问题

提出了一种采用类内迁移学习的异源图像匹配网络(PairsNet)。该网络由特征提取子网络和匹配度量子网络两部分组成。特征提取子网络中存在4条卷积神经网络分支

其通过卷积神经网络分支提取出红外图像和可见光图像的特征。将可见光图像作为源域、红外图像作为目标域进行迁移学习

通过减小两个域中样本特征的类内最大均值差异距离

实现了源域和目标域对应图像类别上精准的样本特征分布对齐。匹配度量子网络使用2个全连接层和1个softmax层进行串联

评估出异源图像特征的匹配度。构建了红外和可见光图像数据集

进行端到端的训练和测试。结果表明:与当前使用预训练模型微调的方法相比

PairsNet的准确率提升了10.54%

可见光图像匹配网络的能力可以有效迁移到异源图像匹配网络。

Abstract

To solve the problem of imaging difference and too few training samples in infrared-visible images matching

a matching network based on intra-class transfer learning is proposed. The network consists of feature extraction subnetwork and metric subnetwork. Owing to four convolutional neural network(CNN)branches in feature extraction subnetwork

the proposed network is referred to as PairsNet for short. The CNN branches extract infrared and visible image features. The visible image is used as the source domain and the infrared image as the target domain. By reducing the maximum mean discrepancy distance of intra-class samples in the two domains

more accurate sample distribution alignment is achieved in the intra-class feature space. The metric subnetwork uses two full connection layers and one softmax layer in series to evaluate infrared-visible image matching performance. Infrared and visible image data sets are built for end-to-end training and testing. The experimental results show

关键词

Keywords

references

李露, 蒋宏, 任章. 红外与可见光复合寻的制导中的快速图像配准方法 [J]. 哈尔滨工程大学学报, 2009, 30(2): 228-232.

LI Lu, JIANG Hong, REN Zhang. A fast image registration algorithm for compound IR/visible light homing guidance [J]. Journal of Harbin Engineering University, 2009, 30(2): 228-232.

LI H, DING W, CAO X, et al. Image registration and fusion of visible and infrared integrated camera for medium-altitude unmanned aerial vehicle remote sensing [J]. Remote Sensing, 2017, 9(5): 441.

叶华, 朱明旱, 王日兴. 红外和可见光图像互补融合的运动目标检测方法 [J]. 红外技术, 2015, 37(8): 648-654.

YE Hua, ZHU Minghan, WANG Rixing. Fusion of complementary information from infrared and visual image for moving object detection [J]. Infrared Technology, 2015, 37(8): 648-654.

CHENG K-S, LIN H-Y. Automatic target recognition by infrared and visible image matching [C]∥14th IAPR International Conference on Machine Vision Applications. Piscataway, NJ, USA: IEEE, 2015: 312-315.

SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition [EB/OL]. [2019-08-15]. https: ∥arxiv.org/pdf/1409. 1556.pdf.

ZHAO F, HUANG Q, GAO W. Image matching by normalized cross-correlation [C]∥ IEEE International Conference on Acoustics, Speech, and Signal Processing. Piscataway, NJ, USA: IEEE, 2006: 729-732.

MAES F, COLLIGNON A, VANDERMEULEN D, et al. Multi-modality image registration by maximization of mutual information [C]∥Proceedings of the Workshop on Mathematical Methods in Biomedical Image Analysis. Piscataway, NJ, USA: IEEE, 1996: 14-22.

SIM D-G, KWON O-K, PARK R-H. Object matching algorithms using robust Hausdorff distance measures [J]. IEEE Transactions on Image Processing, 1999, 8(3): 425-429.

LOWE D G. Distinctive image features from scale-invariant keypoints [J]. International Journal of Computer Vision and Image Understanding, 2004, 60(2): 91-110.

BAY H, TUYTELAARS T, GOOL L J V. SURF: speeded up robust features [J]. Computer Vision and Image Understanding, 2006, 110(3): 404-417.

RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: an efficient alternative to SIFT or SURF [C]∥ IEEE International Conference on Computer Vision. Piscataway, NJ, USA: IEEE, 2011: 2564-2571.

HAN X, LEUNG T, JIA Y, et al. MatchNet: unifying feature and metric learning for patch-based matching [C]∥ IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2015: 3279-3286.

ZAGORUYKO S, KOMODAKIS N. Learning to compare image patches via convolutional neural networks [C]∥ IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2015: 4353-4361.

PAN S J, YANG Q. A survey on transfer learning [J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359.

LONG M, CAO Y, WANG J, et al. Learning transferable features with deep adaptation networks [J]. Journal of Machine Learning Research, 2015, 37: 97-105.

LONG M, WANG J, DING G, et al. Transfer feature learning with joint distribution adaptation [C]∥ IEEE International Conference on Computer Vision. Washington, DC, USA: IEEE, 2013: 2200-2207.

VAN DER MAATEN L, HINTON G E. Visualizing data using t-SNE [J]. Journal of Machine Learning Research, 2008, 9: 2579-2605.

纪利娥, 杨风暴, 王志社, 等. 基于边缘图像和SURF特征的可见光与红外图像的匹配算法 [J]. 红外技术, 2012, 34(11): 629-635.

JI Lie, YANG Fengbao, WANG Zhishe, et al. Visible and infrared image matching algorithm based on edge image and SURF features [J]. Infrared Technology, 2012, 34(11): 629-635.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

An Identification Method for Specific Spacecraft Platform Using Radar Cross Section Sequence

Fault Diagnosis of Rolling Bearings Using Deep Transfer Learning and Adaptive Weighting

Principles, Key Technologies and Emerging Trends of ChatGPT

Lithium Battery State of Charge Estimation Method Based on Transfer Learning

Diffusion Characteristics and Influencing Factors in Case of Hydrogen Leakage from Fuel Cell Vehicles Running in Tunnels

Related Author

李少敏 2

郝培杰 2

苏威 2

牛威 1

LI Naipeng

MA Junxing

SHEN Jianjun

LI Shihao

Related Institution

State Key Laboratory of Astronautic Dynamics, China Xi'an Satellite Control Center

Key Laboratory of Road Construction Technology and Equipment of Ministry of Education, Chang'an University

Key Laboratory of Education Ministry for Modern Design and Rotor-Bearing System, Xi'an Jiaotong University

(MOE Key Lab for Intelligent and Network Security, Xi'an Jiaotong University,,)

State Grid Shaanxi Electric Power Research Institute

⁰