Generating synthetic image data
According to an aspect, a method comprises: generating a 2D projection from a 3D representation of an object, wherein the 2D projection comprises an edgemapped projection of the 3D representation, generating, with a generative adversarial neural network (GAN), and based on the edgemapped projection, a simulated image of the object, wherein the simulated image appears as though the object has been scanned by a detection device, combining the simulated image of the object with a background image to form a synthesized image, wherein the background image was captured by a detection device, and outputting the synthesized image.
1. A method for generating synthesized images performed by a computing system, the method comprising:
obtaining an image of a baggage item captured by a scanning device of a security system;
obtaining an image of an object;
generating, by a generative adversarial network (GAN) and based on the image of the object, a simulated image of the object such that the simulated image of the object comprises one or more image characteristics corresponding to one or more characteristics of images of the object that were captured by scanning devices;
generating a synthesized image based on the image of the baggage item and the simulated image of the object, wherein generating the synthesized image comprises:
selecting an insertion location within the baggage item in the image of the baggage item; and
inserting the simulated image of the object at the insertion location such that the simulated image of the object is within the baggage item in the image of the baggage item, thereby generating the synthesized image;
transforming at least one aspect of the simulated image of the object within the synthesized image in accordance with one or more aspects of the image of the baggage item, thereby generating a transformed synthesized image; and
after generating the transformed synthesized image, training an object detection algorithm with the transformed synthesized image.
2. The method of claim 1 , wherein generating the synthesized image further comprises:
after inserting the simulated image of the object at the insertion location, adding one or more variations to the synthesized image, wherein adding the one or more variations comprises adding at least one of:
lighting, obscuration, illumination, or rotation to the simulated image of the object within the synthesized image.
3. The method of claim 1 , wherein generating the simulated image of the object comprises:
determining a representation of the object in a three-dimensional (3D) virtual coordinate space; and
generating a two-dimensional (2D) slice from the 3D representation of the object.
4. The method of claim 1 , wherein the image of the baggage item comprises at least one background object within the baggage item, and wherein transforming the at least one aspect of the simulated image of the object within the synthesized image comprises overlaying the simulated image of the object within the synthesized image with at least a portion of the at least one background object.
5. The method of claim 1 , wherein selecting the insertion location within the baggage item in the image of the baggage item comprises determining an inside perimeter of the baggage item in the image of the baggage item; and wherein inserting the simulated image of the object at the insertion location such that the simulated image of the object is within the baggage item in the image of the baggage item comprises inserting the simulated image of the object at the insertion location such that the simulated image of the object is completely within the perimeter of the baggage item.
6. The method of claim 1 , wherein the image of the baggage item comprises at least one of: spectral band data or material characteristics that correspond to pixel data of the image of the baggage item.
7. The method of claim 1 , wherein transforming the at least one aspect of the simulated image of the object within the synthesized image comprises:
after generating the synthesized image:
inputting at least a portion of the synthesized image into a second GAN; and
matching, by the second GAN and based on the inputted at least portion of the synthesized image, at least one aspect of the at least portion of the synthesized image to match one or more aspects of the image of the baggage item.
8. A system for generating synthesized images, the system comprising:
at least one processor; and
a non-transitory computer-readable storage medium comprising instructions that are executable by the at least one processor to cause the system to:
obtain an image of a baggage item captured by a scanning device of a security system;
obtain an image of an object;
generate, by a generative adversarial network (GAN) and based on the image of the object, a simulated image of the object such that the simulated image of the object comprises one or more image characteristics corresponding to one or more characteristics of images of the object that were captured by scanning devices;
generate a synthesized image based on the image of the baggage item and the simulated image of the object, wherein to generate the synthesized image comprises:
selecting an insertion location within the baggage item in the image of the baggage item; and
inserting the simulated image of the object at the insertion location such that the simulated image of the object is within the baggage item in the image of the baggage item, thereby generating the synthesized image;
transform at least one aspect of the simulated image of the object within the synthesized image in accordance with one or more aspects of the image of the baggage item, thereby generating a transformed synthesized image; and
after generating the transformed synthesized image, train an object detection algorithm with the transformed synthesized image.
9. The system of claim 8 , wherein the instructions that are executable by the at least one processor to cause the system to generate the synthesized image comprise instructions that are executable by the at least one processor to cause the system to:
after inserting the simulated image of the object at the insertion location, add one or more variations to the synthesized image, wherein adding the one or more variations comprises adding at least one of:
lighting, obscuration, illumination, or rotation to the simulated image of the object within the synthesized image.
10. The system of claim 8 , wherein the instructions that are executable by the at least one processor to cause the system to generate the simulated image of the object comprise instructions that are executable by the at least one processor to cause the system to:
determine a representation of the object in a three-dimensional (3D) virtual coordinate space; and
generate a two-dimensional (2D) slice from the 3D representation of the object.
11. The system of claim 8 , wherein the image of the baggage item comprises at least one background object within the baggage item, and wherein the instructions that are executable by the at least one processor to cause the system to transform the at least one aspect of the simulated image of the object within the synthesized image comprise instructions that are executable by the at least one processor to cause the system to overlay the simulated image of the object within the synthesized image with at least a portion of the at least one background object.
12. The system of claim 8 , wherein selecting the insertion location within the baggage item in the image of the baggage item comprises determining an inside perimeter of the baggage item in the image of the baggage item; and wherein inserting the simulated image of the object at the insertion location such that the simulated image of the object is within the baggage item in the image of the baggage item comprises inserting the simulated image of the object at the insertion location such that the simulated image of the object is completely within the perimeter of the baggage item.
13. The system of claim 8 , wherein the image of the baggage item comprises at least one of: spectral band data or material characteristics that correspond to pixel data of the image of the baggage item.
14. The system of claim 8 , wherein the instructions that are executable by the at least one processor to cause the system to transform the at least one aspect of the simulated image of the object within the synthesized image comprise instructions that are executable by the at least one processor to cause the system to:
after generating the synthesized image:
input at least a portion of the synthesized image into a second GAN; and
match, by the second GAN and based on the inputted at least portion of the synthesized image, at least one aspect of the at least portion of the synthesized image to match one or more aspects of the image of the baggage item.
15. A non-transitory computer-readable storage medium comprising instructions that are executable by at least one processor to cause a computing system to:
obtain an image of a baggage item captured by a scanning device of a security system;
obtain an image of an object;
generate, by a generative adversarial network (GAN) and based on the image of the object, a simulated image of the object such that the simulated image of the object comprises one or more image characteristics corresponding to one or more characteristics of images of the object that were captured by scanning devices;
generate a synthesized image based on the image of the baggage item and the simulated image of the object, wherein to generate the synthesized image comprises:
selecting an insertion location within the baggage item in the image of the baggage item; and
inserting the simulated image of the object at the insertion location such that the simulated image of the object is within the baggage item in the image of the baggage item, thereby generating the synthesized image;
transform at least one aspect of the simulated image of the object within the synthesized image in accordance with one or more aspects of the image of the baggage item, thereby generating a transformed synthesized image; and
after generating the transformed synthesized image, train an object detection algorithm with the transformed synthesized image.
16. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions that are executable by the at least one processor to cause the system to generate the synthesized image comprise instructions that are executable by the at least one processor to cause the system to:
after inserting the simulated image of the object at the insertion location, add one or more variations to the synthesized image, wherein adding the one or more variations comprises adding at least one of:
lighting, obscuration, illumination, or rotation to the simulated image of the object within the synthesized image.
17. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions that are executable by the at least one processor to cause the system to generate the simulated image of the object comprise instructions that are executable by the at least one processor to cause the system to:
determine a representation of the object in a three-dimensional (3D) virtual coordinate space; and
generate a two-dimensional (2D) slice from the 3D representation of the object.
18. The non-transitory computer-readable storage medium of claim 15 , wherein the image of the baggage item comprises at least one background object within the baggage item, and wherein the instructions that are executable by the at least one processor to cause the system to transform the at least one aspect of the simulated image of the object within the synthesized image comprise instructions that are executable by the at least one processor to cause the system to overlay the simulated image of the object within the synthesized image with at least a portion of the at least one background object.
19. The non-transitory computer-readable storage medium of claim 15 , wherein selecting the insertion location within the baggage item in the image of the baggage item comprises determining an inside perimeter of the baggage item in the image of the baggage item; and wherein inserting the simulated image of the object at the insertion location such that the simulated image of the object is within the baggage item in the image of the baggage item comprises inserting the simulated image of the object at the insertion location such that the simulated image of the object is completely within the perimeter of the baggage item.
20. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions that are executable by the at least one processor to cause the system to transform the at least one aspect of the simulated image of the object within the synthesized image comprise instructions that are executable by the at least one processor to cause the system to:
after generating the synthesized image:
input at least a portion of the synthesized image into a second GAN; and
match, by the second GAN and based on the inputted at least portion of the synthesized image, at least one aspect of the at least portion of the synthesized image to match one or more aspects of the image of the baggage item.