Method and system for LULC guided SAR visualization
Optical images in remote sensing are contaminated by cloud cover and bad weather conditions and are only available during the daytime. Whereas SAR images are completely cloud free, independent of weather conditions and can be acquired both during the day and at night. However, due to the speckle effect and side looking imaging mechanism of SAR images, they are not easily interpretable by untrained people. To address this issue, the present disclosure provides a method and system for LULC guided SAR visualization, wherein a GAN is trained to translate SAR images to optical images for visualization. A given SAR image is fed into a first generator of the GAN to obtain LULC map which is then concatenated with the SAR image and fed into a second generator of the GAN to generate an optical image. The LULC map provides semantic information required for generation of more realistic optical image.
1 . A processor implemented method comprising:
receiving, via one or more hardware processors, a Synthetic Aperture RADAR (SAR) image comprising Vertical transmit and Vertical receive (VV) band and Vertical transmit and Horizontal receive (VH) band;
pre-processing, via the one or more hardware processors, the SAR image concatenated with VH/VV band generated from the SAR image;
generating, via the one or more hardware processors, a Land Use Land Cover (LULC) map from the pre-processed image using a first generator of a Generative Adversarial Network (GAN);
concatenating, via the one or more hardware processors, the LULC map with the pre-processed image to obtain a concatenated image; and
generating, via the one or more hardware processors, an optical image from the concatenated image using a second generator of the GAN, wherein the GAN comprises the first generator, the second generator and a discriminator, wherein training of the GAN comprises:
receiving a training dataset comprising (i) a plurality of SAR images, (ii) a plurality of estimated LULC maps associated with each of the plurality of SAR images and (iii) one or more ground truth optical images associated with each of the plurality of SAR images;
training the first generator using the plurality of SAR images and the estimated LULC maps comprised in the training dataset to generate a plurality of LULC maps from the plurality of SAR images;
concatenating the generated plurality of LULC maps with the plurality of SAR images to obtain a plurality of concatenated images; and
jointly training the second generator and the discriminator for a plurality of training epochs.
2 . The method of claim 1 , wherein training the second generator comprises:
training the second generator to generate one or more optical images for each of the plurality of SAR images by using the plurality of concatenated images and the one or more ground truth optical images associated with each of the plurality of SAR images;
classifying the generated one or more optical images from second generator as real or fake by the discriminator; and
updating the second generator based on an adversarial loss and a supervised loss of the second generator.
3 . The method of claim 1 , wherein training the discriminator comprises:
training the discriminator using the one or more ground truth optical images and the generated one or more optical images from second generator to classify an image as real or fake; and
updating the discriminator based on a classification loss of the discriminator.
4 . One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:
receiving a Synthetic Aperture RADAR (SAR) image comprising Vertical transmit and Vertical receive (VV) band and Vertical transmit and Horizontal receive (VH) band;
pre-processing the SAR image concatenated with VH/VV band generated from the SAR image;
generating a Land Use Land Cover (LULC) map from the pre-processed image using a first generator of a Generative Adversarial Network (GAN);
concatenating the LULC map with the pre-processed image to obtain a concatenated image; and
generating an optical image from the concatenated image using a second generator of the GAN, wherein the GAN comprises the first generator, the second generator and a discriminator, wherein training of the GAN comprises:
receiving a training dataset comprising (i) a plurality of SAR images, (ii) a plurality of estimated LULC maps associated with each of the plurality of SAR images and (iii) one or more ground truth optical images associated with each of the plurality of SAR images;
training the first generator using the plurality of SAR images and the estimated LULC maps comprised in the training dataset to generate a plurality of LULC maps from the plurality of SAR images;
concatenating the generated plurality of LULC maps with the plurality of SAR images to obtain a plurality of concatenated images; and
jointly training the second generator and the discriminator for a plurality of training epochs.
5 . A system comprising:
a memory storing instructions;
one or more communication interfaces; and
one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to:
receive a Synthetic Aperture RADAR (SAR) image comprising Vertical transmit and Vertical receive (VV) band and Vertical transmit and Horizontal receive (VH) band;
pre-process the SAR image concatenated with VH/VV band generated from the SAR image;
generate a Land Use Land Cover (LULC) map from the pre-processed image using a first generator of a Generative Adversarial Network (GAN);
concatenate the LULC map with the pre-processed image to obtain a concatenated image; and
generate an optical image from the concatenated image using a second generator of the GAN, wherein the GAN comprises the first generator, the second generator and a discriminator, wherein training of the GAN comprises:
receiving a training dataset comprising (i) a plurality of SAR images, (ii) a plurality of estimated LULC maps associated with each of the plurality of SAR images and (iii) one or more ground truth optical images associated with each of the plurality of SAR images;
training the first generator using the plurality of SAR images and the estimated LULC maps comprised in the training dataset to generate a plurality of LULC maps from the plurality of SAR images;
concatenating the generated plurality of LULC maps with the plurality of SAR images to obtain a plurality of concatenated images; and
jointly training the second generator and the discriminator for a plurality of training epochs.
6 . The system of claim 5 , wherein training the second generator comprises:
training the second generator to generate one or more optical images for each of the plurality of SAR images by using the plurality of concatenated images and the one or more ground truth optical images associated with each of the plurality of SAR images;
classifying the generated one or more optical images from second generator as real or fake by the discriminator; and
updating the second generator based on an adversarial loss and a supervised loss of the second generator.
7 . The system of claim 5 , wherein training the discriminator comprises:
training the discriminator using the one or more ground truth optical images and the generated one or more optical images from second generator to classify an image as real or fake; and
updating the discriminator based on a classification loss of the discriminator.
8 . The one or more non-transitory machine-readable information storage mediums of claim 4 , wherein training the discriminator comprises:
training the discriminator using the one or more ground truth optical images and the generated one or more optical images from second generator to classify an image as real or fake; and
updating the discriminator based on a classification loss of the discriminator.
9 . The one or more non-transitory machine-readable information storage mediums of claim 4 , wherein training the second generator comprises:
training the second generator to generate one or more optical images for each of the plurality of SAR images by using the plurality of concatenated images and the one or more ground truth optical images associated with each of the plurality of SAR images;
classifying the generated one or more optical images from second generator as real or fake by the discriminator; and
updating the second generator based on an adversarial loss and a supervised loss of the second generator.