|
|
|
|
|
Fudan University |
|
|
Despite the advantages of all-weather and all-day high-resolution imaging, synthetic aperture radar (SAR) images are much less viewed and used by general people because human vision is not adapted to microwave scattering phenomenon. However, expert interpreters can be trained by comparing side-byside SAR and optical images to learn the mapping rules from SAR to optical. This paper attempts to develop machine intelligence that is trainable with large-volume co-registered SAR and optical images to translate SAR images to optical version for assisted SAR image interpretation. Reciprocal SAR-optical image translation is a challenging task because it is a raw data translation between two physically very different sensing modalities. Inspired by recent progresses in image translation studies in computer vision, this paper tackles the problem of SAR-optical reciprocal translation with an adversarial network scheme where cascaded residual connections and hybrid L1-GAN loss are employed. It is trained and tested on both spaceborne Gaofen-3 (GF-3) and airborne Uninhabited Airborne Vehicle Synthetic Aperture Radar (UAVSAR) images. Results are presented for datasets of different resolutions and polarizations and compared with other state-of-the-art methods. The Frechet inception distance (FID) is used to quantitatively evaluate the translation performance. The possibility of unsupervised learning with unpaired/unregistered SAR and optical images is also explored. Results show that the proposed translation network works well under many scenarios and it could potentially be used for assisted SAR interpretation.
 |
Shilei Fu, Feng Xu, Ya-Qiu Jin Reciprocal Translation between SAR and Optical Remote Sensing Images with Cascaded-residual adversarial Networks |
|
|
|
|
|
|
|
 |
|
 |
|
Translating images with different resolutions.
The first two rows are chosen from GF-3 dataset, the third row is 6m UAVSAR dataset, and the fourth is from 10m UAVSAR dataset. Images in each row from left to right are the real SAR image ((a1), (a2)) and its translated optical image ((b1), (b2)), the real optical image ((c1), (c2)) and its translated SAR image ((d1), (d2)). |
 |
|
Translaing images with different polarization modes.
Images listed above in each row are (a) the optical ground truth and (b) its translated single-pol SAR image and (c) translated full-pol SAR image, (d) the single-pol SAR ground truth and (e) the optical image translated by single-pol SAR image, (f) the full-pol SAR ground truth and (g) the optical image translated by full-pol SAR image in order. Each row lists a kind of earth surfaces: waters, vegetation, farmlands and buildings. |
 |
|
Comparison of SAR-optical translation by different methods.
Images in each row from left to right are (a) the real optical image, (b) the input SAR image, (c) its translated optical image by CycleGAN, (d) the translated optical image by Pix2Pix and (e) the translated optical image by CRAN. Each row lists a kind of earth surfaces: buildings, buildings, farmlands and roads. |
 |
|
Translated images further refined with unsupervised learning.
Images in each row from left to right are (a) the input SAR image, (b) the translated optical image and (c) the further refined optical image by unsupervised learning, (d) the input optical image and (e) its translated SAR image and (f) the further refined SAR image by unsupervised learning. Each row lists a kind of earth surfaces: waters, vegetation, farmlands and buildings. |
 |
|
Discussion: a contrastive experiment on SAR image segmentation.
Images in each row from left to right are (a) optical ground truth, (b) segmentation ground truth, (c) input SAR image, (d) map segmented from (c), (e) optical image generated from (c) by CRAN and (f) map segmented from (e). For segmentation maps, colors red, green, blue, yellow, and black represent buildings, vegetation, waters, roads, and others respectively. |