Neural network-based image manipulation
An image creation and editing tool can use the data produced from training a neural network to add stylized representations of an object to an image. An object classification will correspond to an object representation, and pixel values for the object representation can be added to, or blended with, the pixel values of an image in order to add a visualization of a type of object to the image. Such an approach can be used to add stylized representations of objects to existing images or create new images based on those representations. The visualizations can be used to create patterns and textures as well, as may be used to paint or fill various regions of an image. Such patterns can enable regions to be filled where image data has been deleted, such as to remove an undesired object, in a way that appears natural for the contents of the image.
1. A computer-implemented method, comprising:
receiving a request to modify an image file, the request indicating an object classification;
obtaining a representation for the object classification, the representation from a neural network trained using a set of images that comprise representations of objects specified by the object classification;
removing image data, corresponding to the representation, from the image file to cause a change in visualization of the representation; and
filing portions in the image file with a matching pattern for a type of object represented in the image file.
2. The computer-implemented method of claim 1 , wherein the change in the visualization is a stylized visualization change that is based at least in part upon a variety of shapes of objects used to train the neural network for the object classification.
3. The computer-implemented method of claim 2 , further comprising:
receiving at least one parameter for use in generating the stylized visualization, the at least one parameter including at least one of a size, aspect ratio, color, style, or weighting.
4. The computer-implemented method of claim 1 , further comprising:
causing additional representations of the type of object to be removed from the image file using the representation for the object classification, the additional representations specified by the request.
5. The computer-implemented method of claim 4 , further comprising:
filling additional portions in the image file where the additional representations are removed.
6. The computer-implemented method of claim 4 , wherein an arrangement of the additional representations of the type of object is random and non-repeating within a specified region.
7. A system, comprising:
at least one processor; and
memory including instructions that, when executed by the at least one processor, cause the system to:
receive a request to modify an image file;
obtain a representation for an object classification from a neural network trained using a set of images that comprise representations of objects associated with the object classification;
remove image data corresponding to the representation from the image file to cause a change in visualization of the image file; and
add data to the image file for a pattern associated with a type of object represented in the image file.
8. The system of claim 7 , wherein the instructions when executed further cause the system to:
receive at least one parameter for use in generating a stylized visualization change for the change in the visualization, the at least one parameter including at least one of a size, aspect ratio, color, style, or weighting.
9. The system of claim 7 , wherein the instructions when executed further cause the system to:
cause additional representations of the type of object to be removed from the image file using the representation for the object classification, the additional representations specified by the request.
10. The system of claim 7 , wherein an arrangement of the additional representations of the type of object is random and non-repeating within a specified region.
11. The system of claim 7 , wherein the instructions when executed further cause the system to:
determine a set of pixel locations associated with a representation of the type of object in the image data;
determine a respective pixel value for an individual pixel location of the set of pixel locations;
determine a corresponding pixel location in the image file for the individual pixel location; and
blend the respective pixel value with the corresponding pixel value for the corresponding pixel location.
12. The system of claim 11 , wherein the instructions when executed further cause the system to:
change a weighting of the respective pixel value with respect to a corresponding pixel value for the corresponding pixel location to affect a visibility of the type of object in the image file when displayed.
13. The system of claim 7 , wherein the instructions when executed further cause the system to:
train neural network models using respective images for each of the object classifications; and
obtain a respective representation for an individual trained neural network model, the respective representation being representative of a type of object associated with the object classification.
14. The system of claim 7 , wherein the instructions when executed further cause the system to:
enable the change in the visualization to be part of a plurality of visualizations having a random pattern that visually corresponds to at least one nearby region in the image file.
15. The system of claim 7 , wherein the instructions when executed further cause the system to:
compare the object classification of the request with a set of object classifications for trained neural networks; and
select a matching object classification that corresponds to the object classification with at least a minimum level of confidence.
16. A non-transitory computer readable storage medium storing one or more sequences of instructions executable by one or more processors to perform a set of steps comprising:
receiving a request to modify an image file;
obtaining a representation for an object classification, the representation from a neural network trained using a set of images that include representations of objects associated with the object classification;
removing image data corresponding to the representation, from the image file to cause a change in visualization in the image file; and
add data to the image file for a type of object represented in the image file.
17. The non-transitory computer readable storage medium of claim 16 , further comprising instructions executable by one or more processors to perform additional steps of:
receiving at least one parameter for use in generating the stylized visualization, the at least one parameter including at least one of a size, aspect ratio, color, style, or weighting.
18. The non-transitory computer readable storage medium of claim 16 , further comprising instructions executable by one or more processors to perform additional steps of:
causing additional representations of the type of object to be removed from the image file using the representation for the object classification, the additional representations specified by the request.
19. The non-transitory computer readable storage medium of claim 16 , further comprising instructions executable by one or more processors to perform additional steps of:
filling additional portions in the image file where the additional representations are removed.
20. The non-transitory computer readable storage medium of claim 16 , further comprising instructions executable by one or more processors to perform additional steps of
determining a set of pixel locations associated with a representation of the type of object in the image data;
determining a respective pixel value for an individual pixel location of the set of pixel locations;
determining a corresponding pixel location in the image file for the individual pixel location; and
blending the respective pixel value with the corresponding pixel value for the corresponding pixel location.