Generation of synthetic high-elevation digital images from temporal sequences of high-elevation digital images
Implementations relate to detecting/replacing transient obstructions from high-elevation digital images, and/or to fusing data from high-elevation digital images having different spatial, temporal, and/or spectral resolutions. In various implementations, first and second temporal sequences of high-elevation digital images capturing a geographic area may be obtained. These temporal sequences may have different spatial, temporal, and/or spectral resolutions (or frequencies). A mapping may be generated of the pixels of the high-elevation digital images of the second temporal sequence to respective sub-pixels of the first temporal sequence. A point in time at which a synthetic high-elevation digital image of the geographic area may be selected. The synthetic high-elevation digital image may be generated for the point in time based on the mapping and other data described herein.
1. A method implemented using one or more processors, comprising:
obtaining a first temporal sequence of high-elevation digital images, wherein the first temporal sequence of high-elevation digital images capture a geographic area at a first temporal frequency and at a first spectral resolution;
obtaining a second temporal sequence of high-elevation digital images, wherein the second temporal sequence of high-elevation digital images capture the geographic area at a second temporal frequency that is less than the first temporal frequency, and at a second spectral resolution that is greater than the first spectral resolution;
selecting a low-resolution reference digital image from the first temporal sequence;
based on a first deviation of ground truth data forming the low-resolution reference digital image from data interpolated from the first temporal sequence of high-elevation digital images, predicting a second deviation from data interpolated from the second temporal sequence of high-elevation digital images; and
based on the low-resolution reference digital image selected from the first temporal sequence, and based on the second deviation, generating a synthetic high-elevation digital image of the geographic area at the second spectral resolution,
wherein the synthetic high-elevation digital image corresponds temporally with the low-resolution reference digital image.
2. The method of claim 1 , wherein the second deviation is proportionate to the first deviation.
3. The method of claim 1 , further comprising generating a mapping of pixels of the high-elevation digital images of the second temporal sequence to respective sub-pixels of the first temporal sequence.
4. The method of claim 3 , wherein the mapping is based on spatial alignment at least some of the pixels of the second temporal sequence that depict a portion of the geographic area with at least some of the respective sub-pixels of the first temporal sequence that depict a same portion of the geographic area.
5. The method of claim 1 , wherein the method further includes:
identifying, across the high-elevation digital images of the second temporal sequence, a plurality of pixel clusters of the second temporal sequence, wherein each pixel cluster of the plurality of pixel clusters comprises pixels with comparable spectral-temporal traces across the second temporal sequence of high-elevation digital images;
wherein the data interpolated from the second temporal sequence comprises one or more centroids calculated from one or more of the pixel clusters.
6. The method of claim 5 , wherein the identifying includes performing K-means clustering on the pixels of the second temporal sequence of high-elevation digital images.
7. The method of claim 6 , wherein each sub-band value of each synthetic pixel of the synthetic high-elevation digital image is calculated based on a deviation of that pixel from a centroid of a pixel cluster that contains that pixel.
8. A method implemented using one or more processors, comprising:
obtaining a first temporal sequence of high-elevation digital images, wherein the first temporal sequence of high-elevation digital images capture a geographic area at a first temporal frequency and at a first spatial resolution;
obtaining a second temporal sequence of high-elevation digital images, wherein the second temporal sequence of high-elevation digital images capture the geographic area at a second temporal frequency that is less than the first temporal frequency, and at a second spatial resolution that is greater than the first spatial resolution;
selecting a low-resolution reference digital image from the first temporal sequence;
based on a first deviation of ground truth data forming the low-resolution reference digital image from data interpolated from the first temporal sequence of high-elevation digital images, predicting a second deviation from data interpolated from the second temporal sequence of high-elevation digital images; and
based on the low-resolution reference digital image selected from the first temporal sequence, and based on the second deviation, generating a synthetic high-elevation digital image of the geographic area at the second spatial resolution,
wherein the synthetic high-elevation digital image corresponds temporally with the low-resolution reference digital image.
9. The method of claim 8 , wherein the second deviation is proportionate to the first deviation.
10. The method of claim 8 , further comprising generating a mapping of pixels of the high-elevation digital images of the second temporal sequence to respective sub-pixels of the first temporal sequence.
11. The method of claim 10 , wherein the mapping is based on spatial alignment at least some of the pixels of the second temporal sequence that depict a portion of the geographic area with at least some of the respective sub-pixels of the first temporal sequence that depict a same portion of the geographic area.
12. The method of claim 8 , wherein the method further includes:
identifying, across the high-elevation digital images of the second temporal sequence, a plurality of pixel clusters of the second temporal sequence, wherein each pixel cluster of the plurality of pixel clusters comprises pixels with comparable spectral-temporal traces across the second temporal sequence of high-elevation digital images;
wherein the data interpolated from the second temporal sequence comprises one or more centroids calculated from one or more of the pixel clusters.
13. The method of claim 12 , wherein the identifying includes performing K-means clustering on the pixels of the second temporal sequence of high-elevation digital images.
14. The method of claim 13 , wherein each sub-band value of each synthetic pixel of the synthetic high-elevation digital image is calculated based on a deviation of that pixel from a centroid of a pixel cluster that contains that pixel.
15. A system comprising one or more processors and memory storing instructions that, in response to execution of the instructions by the one or more processors, cause the one or more processors to:
obtain a first temporal sequence of high-elevation digital images, wherein the first temporal sequence of high-elevation digital images capture a geographic area at a first temporal frequency and at a first spatial resolution;
obtain a second temporal sequence of high-elevation digital images, wherein the second temporal sequence of high-elevation digital images capture the geographic area at a second temporal frequency that is less than the first temporal frequency, and at a second spatial resolution that is greater than the first spatial resolution;
select a low-resolution reference digital image from the first temporal sequence;
based on a first deviation of ground truth data forming the low-resolution reference digital image from data interpolated from the first temporal sequence of high-elevation digital images, predict a second deviation from data interpolated from the second temporal sequence of high-elevation digital images; and
based on the low-resolution reference digital image selected from the first temporal sequence, and based on the second deviation, generate a synthetic high-elevation digital image of the geographic area at the second spatial resolution,
wherein the synthetic high-elevation digital image corresponds temporally with the low-resolution reference digital image.
16. The system of claim 15 , wherein the second deviation is proportionate to the first deviation.
17. The system of claim 15 , further comprising instructions to generate a mapping of pixels of the high-elevation digital images of the second temporal sequence to respective sub-pixels of the first temporal sequence.