Automated color recommendations for images based on object features
Various disclosed embodiments are directed to automatically generating a score indicative of a recommendation for applying a color value to content in an image based on extracting features from one or more objects in the image. In other words, particular embodiments map currently detected objects in an image to objects of the same class of historical images, which are then mapped to their corresponding background colors in order to recommend a particular background color of an image. In an illustrative example, if a particular set of Christmas objects, such as a reindeer object and Santa object, have historically been included in a red background, particular embodiments recommend a red color for the background of a currently processed image that includes the same reindeer and Santa objects.
1 . A system comprising:
at least one computer processor; and
one or more computer storage media storing computer-useable instructions that, when used by the at least one computer processor, cause the at least one computer processor to perform operations comprising:
receiving first image, the first image including first content and second content;
detecting, in the first content of the first image, a first object of a first class, the first class excluding any color features;
and
based on the detected first object in the first content being the first class, automatically generating one or more scores indicative of one or more recommendations for applying one or more color values to the second content.
2 . The system of claim 1 , wherein the one or more operations further comprising: mapping the first object of the first class to at least a second object of the first class of a second image, and wherein the automatic generating of the one or more scores indicative of one or more recommendations for applying one or more color values to the second content is further based on the mapping of the first object of the first class to at least the second object of the first class of the second image.
3 . The system of claim 2 , wherein the generating of the one or more scores is based on training a machine learning model via user-tagged images, the user-tagged images corresponding to images that users modified by changing color of the images, and wherein the second image is included in the user-tagged images.
4 . The system of claim 1 , wherein the generating of the one or more scores is based on using a multi-output Convolutional Neural Network where a first set of outputs represent a first predetermined quantity of colors of objects and a second set of outputs represent a second predetermined quantity of colors of a background.
5 . The system of claim 1 , wherein detecting of the first object includes:
detecting, via one or more bounding boxes, a location of the first object; and
classifying the first object as belonging to the first class.
6 . The system of claim 5 , wherein the operations further comprising:
in response to the detecting of the location and the classifying, detecting boundaries of the first object; and
in response to the detecting of the boundaries, causing the first object to be highlighted at a user interface with a pixel value.
7 . The system of claim 1 , wherein the operations further comprising:
receiving the second image, the second image includes a second set of one or more objects and a background;
detecting the second set of one or more objects; and
removing the background such that only the second set of one or more objects are present in the second image.
8 . The system of claim 7 , wherein the operations further comprising:
extracting color values from the second image;
comparing each color value, of the color values, with a respective predetermined color bucket, of a plurality of predetermined color buckets;
based on the comparing, generating a similarity score for each predetermined color bucket, the similarity score is indicative of a measure of similarity between each color value of the color values and each predetermined color bucket of the plurality of predetermined color buckets;
based at least in part on the generating of the similarity score, ranking each predetermined color bucket of the plurality of predetermined color buckets;
based at least in part on the ranking of each predetermined color bucket, associating the second image with at least a portion of the plurality of predetermined buckets; and
based at least in part on the associating, training a machine learning model by learning weights associated with the color values, and wherein the generating of the one or more scores is based at least in part on the training of the machine learning model.
9 . The system of claim 1 , wherein the operations further comprising:
extracting a second set of features from the first object, the second set of features including color features of the object; and
based further on the color features of the first object, generating the one or more scores indicative of the one or more recommendations for applying the one or more color values to the second content.
10 . A computer-implemented method comprising:
receiving an input image, the input image including first content and second content;
extracting a first set of one or more features from the first content, the one or more features representing a spatial orientation of elements within the first content, wherein the one or more features excluding any color features in the first content; and
based on the spatial orientation of the elements within the first content, automatically generating one or more scores indicative of one or more recommendations for applying one or more color values to the second content.
11 . The computer-implemented method of claim 10 , wherein the first content includes one of: a first set of one or more objects of the input image and a first background of the input image, and wherein the second content includes one of: a second set of one or more objects of the input image and the first background of the input image.
12 . The computer-implemented method of claim 10 , wherein the generating of the one or more scores is based on training a machine learning model via user-tagged images, the user-tagged images corresponding to images that users modified by changing color of the images.
13 . The computer-implemented method of claim 10 , wherein the generating of the one or more scores is based on using a multi-output Convolutional Neural Network where a first set of outputs represent a first predetermined quantity of colors of objects and a second set of outputs represent a second predetermined quantity of colors of a background.
14 . The computer-implemented method of claim 10 , wherein the extracting of the first set of one or more features from the first content is based on one of: detecting, via object detection, a first set of one or more objects or performing image classification of the input image.
15 . The computer-implemented method of claim 14 , further comprising:
in response to the extracting, causing the first set of one or more objects to be highlighted at a user interface with a pixel value.
16 . The computer-implemented method of claim 10 , further comprising:
in response to the extracting of the first set of one or more features from the first content, removing a background such that only a first set of one or more objects are present in the input image;
comparing a vector representing the first set of one or more features with respective vectors, each respective vector representing a second set of one or more features of objects indicated in training images;
based on the comparing, generating a similarity score for each respective vector, the similarity score is indicative of a measure of similarity between the first set of one or more features and the second set of one or more features; and
based on the similarity score exceeding a threshold score for a first set of training images, identifying a background color of the first set of training images, wherein the generating one or more scores indicative of one or more recommendations for generating one or more color values for the second content is based on the identifying of the background color.
17 . The computer-implemented method of claim 16 , further comprising:
based on the identifying of the background color, filling in the background with one of the identified background colors.
18 . The computer-implemented method of claim 10 , further comprising:
extracting a second set of features from the first set of one or more objects, the second set of features including color features of the first set of one or more objects; and
based further on the color features of the first set of one or more objects, generating the one or more scores indicative of the one or more recommendations for applying the one or more color values to the second content.
19 . A computerized system, the system comprising:
an object detector means for receiving an input image, the input image including first content and second content;
wherein the object detector means is further for detecting a first set of one or more objects in the first content of the input image by detecting, via one or more bounding boxes, a location of the first set of one or more objects and classifying the first set of one or more objects to a first class;
determining a boundary of the one or more objects; and
a color recommending means for automatically generating, in response to the detecting, one or more scores indicative of one or more recommendations for applying one or more color values to the second content based on the first class of the first set of one or more objects and the determining of the boundary of the one or more objects.
20 . The system of claim 19 , wherein the second content includes one of: a second set of one or more objects of the input image and a background of the input image.