Semisupervised Remote Sensing Image Fusion Using Multiscale Conditional Generative Adversarial Network with Siamese Structure

Xin Jin, Shanshan Huang, Qian Jiang, Shin Jye Lee, Liwen Wu, Shaowen Yao

Research output: Contribution to journalArticlepeer-review

9 Scopus citations

Abstract

Remote sensing image fusion (RSIF) can generate an integrated image with high spatial and spectral resolution. The fused remote sensing image is conducive to applications including disaster monitoring, ecological environment investigation, and dynamic monitoring. However, most existing deep learning based RSIF methods require ground truths (or reference images) to train a model, and the acquisition of ground truths is a difficult problem. To address this, we propose a semisupervised RSIF method based on the multiscale conditional generative adversarial networks by combining the multiskip connection and pseudo-Siamese structure. This new method can simultaneously extract the features of panchromatic and multispectral images to fuse them without a ground truth; the adopted multiskip connection contributes to presenting image details. In addition, we propose a composite loss function, which combines the least squares loss, L1 loss, and peak signal-to-noise ratio loss to train the model; the composite loss function can help to retain the spatial details and spectral information of the source images. Moreover, we verify the proposed method by extensive experiments, and the results show that the new method can achieve outstanding performance without relying on the ground truth.

Original languageEnglish
Article number9461404
Pages (from-to)7066-7084
Number of pages19
JournalIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Volume14
DOIs
StatePublished - Jun 2021

Keywords

  • Conditional generative adversarial network (cGAN)
  • deep learning (DL)
  • image fusion
  • loss function
  • remote sensing image fusion (RSIF)

Fingerprint

Dive into the research topics of 'Semisupervised Remote Sensing Image Fusion Using Multiscale Conditional Generative Adversarial Network with Siamese Structure'. Together they form a unique fingerprint.

Cite this