Controlled image manipulations using semantic labels
Systems and methods herein describe an image generation system that accesses a semantic map and satellite image, manipulates the semantic map image, trains a machine learning framework using a set of map and satellite image pairs, uses the trained machine learning framework to generate a manipulated satellite image based on the manipulated semantic map, generates a blended satellite image based on a combination of the manipulated satellite image data and the originally accessed satellite image data, and stores the blended satellite image.
1 . A method comprising:
accessing map image data that comprises a plurality of pixels associated with a plurality of map item labels;
accessing satellite image data associated with the map image data;
manipulating at least a portion of map item labels of the plurality of map item labels to generate manipulated map image data;
training a machine learning framework using a set of map and satellite image pairs, the set of map and satellite image pairs depicting a geographic region visually similar to the accessed map image data and the accessed satellite image data;
using the trained machine learning framework to generate manipulated satellite image data based on the manipulated map image data;
generating a blended satellite image based on a combination of the manipulated satellite image data and the accessed satellite image data; and
storing the blended satellite image.
2 . The method of claim 1 , wherein the machine learning framework is a generative adversarial network (GAN).
3 . The method of claim 1 , wherein the manipulating at least the portion of map item labels further comprises:
at least one of removing, inserting or replacing an object depicted in the accessed map image data from the accessed map image data.
4 . The method of claim 1 , wherein the set of map and satellite image pairs comprises only the accessed map image data and the accessed satellite image data.
5 . The method of claim 1 , wherein the blended satellite image is generated based on the combination of the manipulated satellite image data and a second satellite image from the set of map and satellite image pairs.
6 . The method of claim 1 , wherein the geographic region visually similar to the accessed map image data and the accessed satellite image data comprises a same set of natural geographic features and engineered geographic features as the accessed map image data and the accessed satellite image data.
7 . The method of claim 1 , wherein the set of map and satellite image pairs comprise images that are captured from location coordinates that are within a predefined radius of location coordinates associated with the accessed map image data and the accessed satellite image data.
8 . A system comprising:
one or more hardware processors; and
a memory storing instructions that, when executed by the one or more hardware processors, configure the system to perform operations comprising:
accessing map image data that comprises a plurality of pixels associated with a plurality of map item labels;
accessing satellite image data associated with the map image data;
manipulating at least a portion of map item labels of the plurality of map item labels to generate manipulated map image data;
training a machine learning framework using a set of map and satellite image pairs, the set of map and satellite image pairs depicting a geographic region visually similar to the accessed map image data and the accessed satellite image data;
using the trained machine learning framework to generate manipulated satellite image data based on the manipulated map image data;
generating a blended satellite image based on a combination of the manipulated satellite image data and the accessed satellite image data; and
storing the blended satellite image.
9 . The system of claim 8 , wherein the machine learn framework is a generative adversarial network (GAN).
10 . The system of claim 8 , wherein the manipulating at least the portion of map item labels further comprises:
at least one of removing, inserting or replacing an object depicted in the accessed map image data from the accessed map image data.
11 . The system of claim 8 , wherein the set of map and satellite image pairs comprises only the accessed map image data and the accessed satellite image data.
12 . The system of claim 8 , wherein the blended satellite image is generated based on the combination of the manipulated satellite image data and a second satellite image from the set of map and satellite image pairs.
13 . The system of claim 8 , wherein the geographic region visually similar to the accessed map image data and the accessed satellite image data comprises a same set of natural geographic features and engineered geographic features as the accessed map image data and the accessed satellite image data.
14 . The system of claim 8 , wherein the set of map and satellite image pairs comprise images that are captured from location coordinates that are within a predefined radius of location coordinates associated with the accessed map image data and the accessed satellite image data.
15 . A non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium including instructions that when executed by a computer, cause the computer to perform operations comprising:
accessing map image data that comprises a plurality of pixels associated with a plurality of map item labels;
accessing satellite image data associated with the map image data;
manipulating at least a portion of map item labels of the plurality of map item labels to generate manipulated map image data;
training a machine learning framework using a set of map and satellite image pairs, the set of map and satellite image pairs depicting a geographic region visually similar to the accessed map image data and the accessed satellite image data;
using the trained machine learning framework to generate manipulated satellite image data based on the manipulated map image data;
generating a blended satellite image based on a combination of the manipulated satellite image data and the accessed satellite image data; and
storing the blended satellite image.
16 . The non-transitory computer-readable storage medium of claim 15 , wherein the machine learning framework is a generative adversarial network (GAN).
17 . The non-transitory computer-readable storage medium of claim 15 , wherein the manipulating at least the portion of map item labels further comprises:
at least one of removing, inserting or replacing an object depicted in the accessed map image data from the accessed map image data.
18 . The non-transitory computer-readable storage medium of claim 15 , wherein the set of map and satellite image pairs comprises only the accessed map image data and the accessed satellite image data.
19 . The non-transitory computer-readable storage medium of claim 15 , wherein the blended satellite image is generated based on the combination of the manipulated satellite image data and a second satellite image from the set of map and satellite image pairs.
20 . The non-transitory computer-readable storage medium of claim 15 , wherein the geographic region visually similar to the accessed map image data and the accessed satellite image data comprises a same set of natural geographic features and engineered geographic features as the accessed map image data and the accessed satellite image data.