采用类内迁移学习的红外/可见光异源图像匹配

毛远宏; 贺占庄; 马钟; 毕瑞星; 王竹平

doi:10.7652/xjtuxb202001007

您当前的位置：

首页 >

文章列表页 >

采用类内迁移学习的红外/可见光异源图像匹配

更新时间：2025-07-09

- 采用类内迁移学习的红外/可见光异源图像匹配
- Infrared-Visible Heterogenous Image Matching Based on Intra-Class Transfer Learning
- 西安交通大学学报 2020年54卷第1期页码：49-55
- 作者机构：
  
  西安微电子技术研究所,西安,710065
- 作者简介：
- 基金信息：
  
  : 2019-06-14。作者简介: 毛远宏(1980—),男,博士生;贺占庄(通信作者),男,研究员,博士生导师。基金项目: 国家自然科学基金青年科学基金资助项目(61702413)
- DOI：10.7652/xjtuxb202001007
  中图分类号： TP391
- 网络首发：2020-01-10，
  
  纸质出版：2020
- 稿件说明：
移动端阅览
毛远宏, 贺占庄, 马钟, 等. 采用类内迁移学习的红外/可见光异源图像匹配[J]. 西安交通大学学报, 2020,54(1):49-55.

Infrared-Visible Heterogenous Image Matching Based on Intra-Class Transfer Learning[J]. 2020, 54(1): 49-55.
毛远宏, 贺占庄, 马钟, 等. 采用类内迁移学习的红外/可见光异源图像匹配[J]. 西安交通大学学报, 2020,54(1):49-55. DOI： 10.7652/xjtuxb202001007.

Infrared-Visible Heterogenous Image Matching Based on Intra-Class Transfer Learning[J]. 2020, 54(1): 49-55. DOI： 10.7652/xjtuxb202001007.

摘要

为解决异源图像匹配中样本量过少和成像原理不同导致成像差异的问题

提出了一种采用类内迁移学习的异源图像匹配网络(PairsNet)。该网络由特征提取子网络和匹配度量子网络两部分组成。特征提取子网络中存在4条卷积神经网络分支

其通过卷积神经网络分支提取出红外图像和可见光图像的特征。将可见光图像作为源域、红外图像作为目标域进行迁移学习

通过减小两个域中样本特征的类内最大均值差异距离

实现了源域和目标域对应图像类别上精准的样本特征分布对齐。匹配度量子网络使用2个全连接层和1个softmax层进行串联

评估出异源图像特征的匹配度。构建了红外和可见光图像数据集

进行端到端的训练和测试。结果表明:与当前使用预训练模型微调的方法相比

PairsNet的准确率提升了10.54%

可见光图像匹配网络的能力可以有效迁移到异源图像匹配网络。

Abstract

To solve the problem of imaging difference and too few training samples in infrared-visible images matching

a matching network based on intra-class transfer learning is proposed. The network consists of feature extraction subnetwork and metric subnetwork. Owing to four convolutional neural network(CNN)branches in feature extraction subnetwork

the proposed network is referred to as PairsNet for short. The CNN branches extract infrared and visible image features. The visible image is used as the source domain and the infrared image as the target domain. By reducing the maximum mean discrepancy distance of intra-class samples in the two domains

more accurate sample distribution alignment is achieved in the intra-class feature space. The metric subnetwork uses two full connection layers and one softmax layer in series to evaluate infrared-visible image matching performance. Infrared and visible image data sets are built for end-to-end training and testing. The experimental results show

关键词

Keywords

references

李露, 蒋宏, 任章. 红外与可见光复合寻的制导中的快速图像配准方法 [J]. 哈尔滨工程大学学报, 2009, 30(2): 228-232.

LI Lu, JIANG Hong, REN Zhang. A fast image registration algorithm for compound IR/visible light homing guidance [J]. Journal of Harbin Engineering University, 2009, 30(2): 228-232.

LI H, DING W, CAO X, et al. Image registration and fusion of visible and infrared integrated camera for medium-altitude unmanned aerial vehicle remote sensing [J]. Remote Sensing, 2017, 9(5): 441.

叶华, 朱明旱, 王日兴. 红外和可见光图像互补融合的运动目标检测方法 [J]. 红外技术, 2015, 37(8): 648-654.

YE Hua, ZHU Minghan, WANG Rixing. Fusion of complementary information from infrared and visual image for moving object detection [J]. Infrared Technology, 2015, 37(8): 648-654.

CHENG K-S, LIN H-Y. Automatic target recognition by infrared and visible image matching [C]∥14th IAPR International Conference on Machine Vision Applications. Piscataway, NJ, USA: IEEE, 2015: 312-315.

SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition [EB/OL]. [2019-08-15]. https: ∥arxiv.org/pdf/1409. 1556.pdf.

ZHAO F, HUANG Q, GAO W. Image matching by normalized cross-correlation [C]∥ IEEE International Conference on Acoustics, Speech, and Signal Processing. Piscataway, NJ, USA: IEEE, 2006: 729-732.

MAES F, COLLIGNON A, VANDERMEULEN D, et al. Multi-modality image registration by maximization of mutual information [C]∥Proceedings of the Workshop on Mathematical Methods in Biomedical Image Analysis. Piscataway, NJ, USA: IEEE, 1996: 14-22.

SIM D-G, KWON O-K, PARK R-H. Object matching algorithms using robust Hausdorff distance measures [J]. IEEE Transactions on Image Processing, 1999, 8(3): 425-429.

LOWE D G. Distinctive image features from scale-invariant keypoints [J]. International Journal of Computer Vision and Image Understanding, 2004, 60(2): 91-110.

BAY H, TUYTELAARS T, GOOL L J V. SURF: speeded up robust features [J]. Computer Vision and Image Understanding, 2006, 110(3): 404-417.

RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: an efficient alternative to SIFT or SURF [C]∥ IEEE International Conference on Computer Vision. Piscataway, NJ, USA: IEEE, 2011: 2564-2571.

HAN X, LEUNG T, JIA Y, et al. MatchNet: unifying feature and metric learning for patch-based matching [C]∥ IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2015: 3279-3286.

ZAGORUYKO S, KOMODAKIS N. Learning to compare image patches via convolutional neural networks [C]∥ IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2015: 4353-4361.

PAN S J, YANG Q. A survey on transfer learning [J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359.

LONG M, CAO Y, WANG J, et al. Learning transferable features with deep adaptation networks [J]. Journal of Machine Learning Research, 2015, 37: 97-105.

LONG M, WANG J, DING G, et al. Transfer feature learning with joint distribution adaptation [C]∥ IEEE International Conference on Computer Vision. Washington, DC, USA: IEEE, 2013: 2200-2207.

VAN DER MAATEN L, HINTON G E. Visualizing data using t-SNE [J]. Journal of Machine Learning Research, 2008, 9: 2579-2605.

纪利娥, 杨风暴, 王志社, 等. 基于边缘图像和SURF特征的可见光与红外图像的匹配算法 [J]. 红外技术, 2012, 34(11): 629-635.

JI Lie, YANG Fengbao, WANG Zhishe, et al. Visible and infrared image matching algorithm based on edge image and SURF features [J]. Infrared Technology, 2012, 34(11): 629-635.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

利用雷达散射截面积序列的特定航天器工作平台分类识别方法

采用深度迁移学习与自适应加权的滚动轴承故障诊断

ChatGPT的工作原理、关键技术及未来发展趋势

采用迁移学习的小样本锂电池荷电状态估算方法

语义区分和子图整合驱动的知识图谱表示学习方法