Visual attribute determination for content selection
Content can be located for items that are stylistically similar to an item of interest. The item of interest can be represented in a query image, which is analyzed to determine one or more regions having an item represented therein. The classification of the item is determined, enabling identification of a trained model to be used to process image data for the region(s) of the query image. The trained model outputs a set of attributes, relating to visual or stylistic attributes, and corresponding confidence or prominence values for the attributes. These attributes and values can be compared against a data repository to locate items determined to be similar based on corresponding attributes and values. A similarity determination algorithm can identify similar items and rank those items by similarity. Content for the most similar items is returned as a result for the query image.
1. A computer-implemented method, comprising:
receiving a query image including a representation of an item of interest;
locating the representation of the item of interest in the query image;
determining an item type of the item or interest;
processing the representation of the item using a trained machine learning model corresponding to the item type;
obtaining, from the trained machine learning model, a set of attributes and confidence values, the attributes including stylistics attributes exhibited by the representation of the item in the query image;
determining, using the set of attributes and confidence values, similarity scores for a set of similar items to the item of interest;
determining a ranking of the set of similar items, the ranking based at least in part on the similarity scores; and
providing content corresponding to at least a subset of the similar items, the subset based at least in part on the ranking.
2. The computer-implemented method of claim 1 , further comprising:
processing the query image using a localizer algorithm to determine a region of the query image including the representation of the item of interest.
3. The computer-implemented method of claim 2 , further comprising:
processing image data for the region using a trained classifier to determine the item type.
4. The computer-implemented method of claim 1 , wherein the stylistic attributes include at least one of a color, a pattern, a cut, a length, a shape, a silhouette, a neckline, a hemline, or an occasion type of the item of interest.
5. The computer-implemented method of claim 1 , further comprising:
training the trained machine learning model using a set of annotated images including items of the item type, wherein similarity of the items to the item of interest are able to be determined using a similarity determination algorithm accepting as input the attributes and confidence values.
6. A computer-implemented method, comprising:
processing an image using a trained model to produce a set of attributes representative of an item represented in the image, the attributes relating to at least one of visual attributes or stylistic attributes;
determining weighted relationships among the set of attributes for the item;
comparing the weighted relationships of the attributes against attribute data stored for items having been previously processed to identify a set of stylistically similar items having similar weighted relationships of attributes;
determining respective similarity scores for the set of stylistically similar items with respect to the item, the respective similarity scores based at least in part on the set of attributes;
ranking the stylistically similar items by the respective similarity scores;
determining a subset of the stylistically similar items based in part upon highest ranking by the respective similarity scores; and
providing content associated with at least the subset of the stylistically similar items.
7. The computer-implemented method of claim 6 , further comprising:
processing the image using a localizer algorithm to determine a region of the query image including the representation of the item.
8. The computer-implemented method of claim 7 , further comprising:
processing image data for the region using a trained classifier to determine an item type for the item.
9. The computer-implemented method of claim 8 , further comprising:
determining the weighted relationships by processing the representation of the item using a trained machine learning model corresponding to the item type.
10. The computer-implemented method of claim 6 , further comprising:
training the trained machine learning model using a set of annotated images including items of the item type, wherein similarity of the items to the item of interest are able to be determined using a similarity determination algorithm accepting as input the attributes and confidence values.
11. The computer-implemented method of claim 6 , wherein the stylistic attributes include at least one of a color, a pattern, a cut, a length, a shape, a silhouette, a neckline, a hemline, or an occasion type of the item of interest.
12. A system, comprising:
at least one processor; and
memory storing instructions that, when executed by the at least one processor, cause the system to:
process an image using a trained model to produce a set of attributes representative of an item represented in the image, the attributes relating to at least one of visual attributes or stylistic attributes of the item;
determine weighted relationships among the set of attributes for the item;
compare the weighted relationships of the attributes against attribute data stored for items having been previously processed to identify a set of stylistically similar items having similar weighted relationships of attributes;
determine respective similarity scores for the set of stylistically similar items with respect to the item, the respective similarity scores based at least in part on the set of attributes;
rank the stylistically similar items by the respective similarity scores;
determine a subset of the stylistically similar items based in part upon highest ranking by the respective similarity scores; and
provide content associated with at least a subset of the stylistically similar items.
13. The system of claim 12 , wherein the instructions when executed further cause the system to:
process the image using a localizer algorithm to determine a region of the query image including the representation of the item.
14. The system of claim 13 , wherein the instructions when executed further cause the system to:
process image data for the region using a trained classifier to determine an item type for the item.
15. The computer-implemented method of claim 14 , further comprising:
determine the weighted relationships by processing the representation of the item using a trained machine learning model corresponding to the item type.
16. The system of claim 12 , wherein the instructions when executed further cause the system to:
train the trained machine learning model using a set of annotated images including items of the item type, wherein similarity of the items to the item of interest are able to be determined using a similarity determination algorithm accepting as input the attributes and confidence values.