A Relativistic Average Generative Adversarial Network for Pan-Sharpening

doi:10.7652/xjtuxb202203006

您当前的位置：

首页 >

文章列表页 >

A Relativistic Average Generative Adversarial Network for Pan-Sharpening

更新时间：2025-07-09

- A Relativistic Average Generative Adversarial Network for Pan-Sharpening
- Vol. 56, Issue 3, Pages: 54-64(2022)
- 作者机构：
  
  长安大学信息工程学院,西安,710000
- 作者简介：
- 基金信息：
- DOI：10.7652/xjtuxb202203006
  CLC： TP391
- Online First：10 March 2022，
  
  Published：2022
- 稿件说明：
移动端阅览
A Relativistic Average Generative Adversarial Network for Pan-Sharpening[J]. 2022, 56(3): 54-64.
DOI：

A Relativistic Average Generative Adversarial Network for Pan-Sharpening[J]. 2022, 56(3): 54-64. DOI： 10.7652/xjtuxb202203006.

摘要

为解决全色锐化过程中对原图像特征提取不足导致融合结果细节信息易丢失

以及图像融合过程中因忽略不同区域的空间特征差异而导致信息冗余等问题

采用深度学习算法

提出一种用于全色锐化的相对平均生成对抗网络(Pan-RaGAN)。在生成器中利用改进的密集块结构对原图像进行特征提取

充分利用原图像各级特征

获取包含了更多细节信息的融合结果; 提出基于空间注意力机制的特征细化模块

用于特征选择

可在保留有效高频信息的同时剔除冗余信息; 利用图像重建模块将细化后的特征与上采样的低分辨率多光谱图像进行融合

以保持光谱信息; 利用相对平均鉴别器改进网络的损失函数

进一步优化融合效果。在高分2号卫星和快鸟卫星图像数据集上的实验结果表明

与已有用于遥感图像全色锐化的生成对抗网络相比

Pan-RaGAN网络的光谱角映射指标平均降低了0.075

验证了Pan-RaGAN网络的有效性。

Abstract

A relativistic average generative adversarial network for pan-sharpening(Pan-RaGAN)based on a deep learning algorithm is proposed to solve the problems that details of fusion result are easy to be lost due to insufficient feature extraction from original image in the pan-sharpening process and information redundancy is caused by ignoring spatial feature difference of different regions in image fusion process. Firstly

the improved dense block structure is utilized to extract the features of the original image in the generator

so as to make full use of the features from different layers of the original image and obtain better fusion results with more details. Secondly

a feature refinement module based on spatial attention mechanism is proposed for feature selection

which can make a better trade-off between retaining effective high-frequency information and eliminating redundant information. Furthermore

the image reconstruction module is utilized to fuse the refined features with the up-sampled low resolution multispectral images to preserve the spectral information. Finally

the relativistic average discriminator is utilized to improve the loss function of the network

and further optimize the fusion effect. Experimental results on Gao Fen-2 dataset and Quick Bird dataset and a comparison with the existing generative adversarial network for remote sensing image pan-sharpening show that the spectral angle mapper index of the proposed Pan-RaGAN network is reduced by 0.075 on average

which verifies the effectiveness of the proposed Pan-RaGAN network.

关键词

Keywords

references

高雪琴, 刘刚, 肖刚, 等. 基于FPDE的红外与可见光图像融合算法 [J]. 自动化学报, 2020, 46(4): 796-804.

GAO Xueqin, LIU Gang, XIAO Gang, et al. Fusion algorithm of infrared and visible images based on FPDE [J]. Acta Automatica Sinica, 2020, 46(4): 796-804.

刘锐, 赵旭, 赵东利, 等. 基于MIM软件行CT-MRI图像融合技术在脑胶质瘤精确放疗中的应用 [J]. 西安交通大学学报(医学版), 2020, 41(5): 668-672.

LIU Rui, ZHAO Xu, ZHAO Dongli, et al. Application of CT-MRI image fusion technology based on MIM software in accurate radiotherapy of glioma [J]. Journal of Xi'an Jiaotong University(Medical Sciences), 2020, 41(5): 668-672.

陈超迁, 孟勇, 杨平吕, 等. 基于边缘增强与光谱特性保持的Pan-sharpening融合模型 [J]. 自动化学报, 2019, 45(2): 374-387.

CHEN Chaoqian, MENG Yong, YANG Pinglv, et al. Pan-sharpening model on account of edge enhancement and spectral signature preservation [J]. Acta Automatica Sinica, 2019, 45(2): 374-387.

李策, 张亚超, 蓝天, 等. 一种高分辨率遥感图像视感知目标检测算法 [J]. 西安交通大学学报, 2018, 52(6): 9-16.

LI Ce, ZHANG Yachao, LAN Tian, et al. An object detection algorithm with visual perception for high-resolution remote sensing images [J]. Journal of Xi'an Jiaotong University, 2018, 52(6): 9-16.

TONG Xinyi, LU Qikai, XIA Guisong, et al. Large-scale land cover classification in Gaofen-2 satellite imagery [C]∥Proceedings of the 2018 IEEE International Geoscience and Remote Sensing Symposium(IGARSS). Piscataway, NJ, USA: IEEE, 2018: 3599-3602.

LIU Yu, CHEN Xun, WANG Zengfu, et al. Deep learning for pixel-level image fusion: recent advances and future prospects [J]. Information Fusion, 2018, 42: 158-173.

MENG Xiangchao, SHEN Huanfeng, LI Huifang, et al. Review of the pansharpening methods for remote sensing images based on the idea of meta-analysis: practical discussion and challenges [J]. Information Fusion, 2019, 46: 102-113.

CARPER W J, LILLESAND T M, KIEFER R W. The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data [J]. Photogrammetric Engineering and Remote Sensing, 1990, 56(4): 459-467.

RAHMANI S, STRAIT M, MERKURJEV D, et al. An adaptive IHS pan-sharpening method [J]. IEEE Geoscience and Remote Sensing Letters, 2010, 7(4): 746-750.

KING R L, WANG Jianwen. A wavelet based algorithm for pan sharpening Landsat 7 imagery [C]∥Proceedings of the 2001 International Geoscience and Remote Sensing Symposium(IGARSS). Piscataway, NJ, USA: IEEE, 2001: 849-851.

LIU J G. Smoothing filter-based intensity modulation: a spectral preserve image fusion technique for improving spatial details [J]. International Journal of Remote Sensing, 2000, 21(18): 3461-3472.

TIAN Xin, CHEN Yuerong, YANG Changcai, et al. A variational pansharpening method based on gradient sparse representation [J]. IEEE Signal Processing Letters, 2020, 27: 1180-1184.

MASI G, COZZOLINO D, VERDOLIVA L, et al. Pansharpening by convolutional neural networks [J]. Remote Sensing, 2016, 8(7): 594.

HE Lin, RAO Yizhou, LI Jun, et al. Pansharpening via detail injection based convolutional neural networks [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(4): 1188-1204.

LIU Xiangyu, LIU Qingjie, WANG Yunhong. Remote sensing image fusion based on two-stream fusion network [J]. Information Fusion, 2020, 55: 1-15.

SHAO Zhenfeng, CAI Jiajun. Remote sensing image fusion with deep convolutional neural network [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(5): 1656-1669.

GASTINEAU A, AUJOL J F, BERTHOUMIEU Y, et al. Generative adversarial network for pansharpening with spectral and spatial discriminators [J/OL]. IEEE Transactions on Geoscience and Remote Sensing, 2021 [2021-05-01]. https: ∥ieeexplore.ieee. org/document/9371303.

LIU Qingjie, ZHOU Huanyu, XU Qizhi, et al. PSGAN: a generative adversarial network for remote sensing image pan-sharpening [J/OL]. IEEE Transactions on Geoscience and Remote Sensing, 2020 [2021-05-01]. https: ∥ieeexplore.ieee.org/docume nt/9306912.

SHAO Zhimin, LU Zexin, RAN Maosong, et al. Residual encoder-decoder conditional generative adversarial network for pansharpening [J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(9): 1573-1577.

JOLICOEUR-MARTINEAU A. The relativistic discriminator: a key element missing from standard GAN [C/OL]∥Proceedings of the 2019 International Conference on Learning Representations. London, UK: ICLR, 2019 [2021-05-01]. https: ∥arxiv.org/pdf/1807.00734.pdf.

WANG Xintao, YU Ke, WU Shixiang, et al. ESRGAN: enhanced super-resolution generative adversarial networks [C]∥Proceedings of the 2018 ECCV Workshops. Cham, Germany: Springer, 2019: 63-79.

MA Jiayi, YU Wei, CHEN Chen, et al. Pan-GAN: an unsupervised pan-sharpening method for remote sensing image fusion [J]. Information Fusion, 2020, 62: 110-120.

ZHOU Huanyu, LIU Qingjie, WANG Yunhong. PGMAN: an unsupervised generative multiadversarial network for pansharpening [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 6316-6327.

HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks [C]∥Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ, USA: IEEE, 2017: 2261-2269.

陈一鸣, 周登文. 基于自适应级联的注意力网络的超分辨重建 [J/OL]. 自动化学报, 2020 [2021-05-01]. https: ∥doi.org/10.16383/j.aas.c200035.

CHEN Yiming, ZHOU Dengwen. Adaptive attention network for image super-resolution [J/OL]. Acta Automatica Sinica, 2020 [2021-05-01]. https: ∥doi. org/10.16383/j.aas.c200035.

HU Yanting, LI Jie, HUANG Yuanfei, et al. Channel-wise and spatial feature modulation network for single image super-resolution [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(11): 3911-3927.

LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution [C]∥Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW). Piscataway, NJ, USA: IEEE, 2017: 1132-1140.

YUHAS R H, GOETZ A F H, BOARDMAN J W. Discrimination among semi-arid landscape endmembers using the spectral angle mapper(SAM)algorithm [C]∥Proceedings of the Summaries of the Third Annual JPL Airborne Geoscience Workshop. Pasadena, CA, USA: JPL, 1992: 147-149.

ZHOU J, CIVCO D L, SILANDER J A. A wavelet transform method to merge Landsat TM and SPOT panchromatic data [J]. International Journal of Remote Sensing, 1998, 19(4): 743-757.

WALD L. Quality of high resolution synthesised images: is there a simple criterion? [C]∥Proceedings of the Third Conference “Fusion of Earth Data: Merging Point Measurements, Raster Maps And Remotely Sensed Images”. Sophia Antipolis, France: SEE/URISCA, 2000: 99-103.

WANG Zhou, BOVIK A C. A universal image quality index [J]. IEEE Signal Processing Letters, 2002, 9(3): 81-84.

ALPARONE L, AIAZZI B, BARONTI S, et al. Multispectral and panchromatic data fusion assessment without reference [J]. Photogrammetric Engineering Remote Sensing, 2008, 74(2): 193-200.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Intelligent Quantitative Non-Destructive Evaluation Method for Delamination of Carbon Fiber Composite Tube with Mirror-Reflection Infrared Thermography

Model Predictive Control-Oriented Deep Koopman Global Linear Modeling Method for Wind Turbines

Dynamic Liquid Level Prediction of Steam Generator under Main Steam Pipe Rupture Accidents

A Pure Convolutional Model Fused with U-net Network for Video Prediction

Application of Interpretable Convolutional Neural Network Enabled by Bi-Damped Wavelets in Bearing Fault Diagnosis

Related Author

YANG Yang

CHEN Hongen

LIU Bofan

TONG Zongfei

PEI Cuixiang

CHEN Zhenmao

TIAN Runze

KOU Peng

Related Institution

School of Aerospace Engineering，Xi’an Jiaotong University

State Key Laboratory for Strength and Vibration of Mechanical Structures，Xi’an Jiaotong University

Shaanxi Engineering Research Center of Nondestructive Testing and Structural Integrity Evaluation，Xi’an Jiaotong University

Northern Branch, Beijing Jingneng International Holding Co., Ltd.

State Key Laboratory of Electrical Insulation and Power Equipment, Xi'an Jiaotong University

⁰