Color adaptation using adversarial training networks
Systems and methods are provided for adapting images from different cameras so that a single trained classifier or an analyzer may be used. The classifier or analyzer operates on images that include a particular color distribution or characteristic. A generative network is used to adapt images from other cameras to have a similar color distribution or characteristic for use by the classifier or analyzer. A generative adversarial process is used to train the generative network.
1. A method for adapting a second sky image from a second camera to be classified with a classifier trained on a plurality of first sky images from a first camera, the method comprising:
acquiring the second sky image from the second camera;
synthesizing from the second sky image, using a generative network having been trained with generative adversarial training, a third sky image, the third sky image including one or more color features that are similar to one or more color features of the plurality of first sky images; and
detecting one or more clouds in the third sky image using the classifier.
2. The method of claim 1 , wherein training the generative network comprises:
synthesizing, by the generative network, a fifth sky image from a fourth sky image acquired from the second camera;
determining, using a discriminative network, how likely the synthesized fifth sky image was acquired by the first camera or how likely the synthesized fifth sky image was synthesized using the generative network; and
adjusting the generative network as a function of the determination.
3. The method of claim 2 , wherein the training further comprises:
adjusting the discriminative network to make the determination more accurate.
4. The method of claim 2 , wherein synthesizing the fifth sky image comprises:
altering one or more color features of the fourth sky image on a pixel by pixel basis.
5. The method of claim 2 , wherein determining is binary; wherein the two binary choices are that the synthesized fifth sky image was acquired by the second device or the synthesized fifth sky image was synthesized using the generative network.
6. The method of claim 2 , wherein adjusting comprises:
altering weights of one or more filters of the generative network.
7. The method of claim 1 , wherein detecting comprises:
segmenting the third sky image using a machine learnt classifier.
8. The method of claim 1 , wherein the second sky image is not paired with any of the plurality of first sky images.
9. A system for adapting sky images the system comprising:
a first camera configured to acquire first sky images;
a second camera configure to acquire second sky images;
a machine learnt generative network configured to generate synthesized images from the second sky images, the synthesized images including different color features than the second sky images; and
a cloud forecaster configured to identify clouds in the synthesized images, the cloud forecaster further configured to predict cloud coverage for a location.
10. The system of claim 9 , wherein the machine learnt generative network is trained using an adversarial network.
11. The system of claim 10 , wherein the adversarial network comprises:
the machine learnt generative network; and
a machine learnt discriminative network configured to classify input images as either synthesized images or first sky images;
wherein the machine learnt generative network is adjusted as a function of the classification.
12. The system of claim 9 , wherein the cloud forecaster is configured to identify clouds using a machine learnt classifier.
13. The system of claim 12 , wherein the machine learnt classifier is trained to segment sky images based on color features of the first sky images from the first camera.