Systems and methods for product search using generated shape descriptors and edge image of source product images
Systems and techniques for a design-aware image search are described. The design-aware image search techniques described herein capture a design on an item to determine additional items with similar or matching designs. An image of the item is used to create an edge image of a design, and shape descriptors are generated describing features of the edge image. These shape descriptors are compared to shape descriptors associated with other images to locate images that have similar or matching designs as compared with the input image. The design-aware image search system may use these relationships to generate a search result with images or products having a design similar to the design on the input image.
1. A system comprising:
one or more processors; and
a memory storing instructions that are executable by the one or more processors to:
generate, for each respective one of a plurality of indications of products, an edge image corresponding to an image associated with each one of the plurality of the indications of the products using a machine learning model to detect a feature depicted in the image, the edge image including lines identifying edges of the feature;
generate shape descriptors describing one or more pixel configurations or vector configurations for each respective one of the edge images;
receive, from a client device and via a network, an input image including a design as part of a search input;
generate an edge image for the design based on the input image using the machine learning model;
generate shape descriptors for the design based on the edge image for the design;
locate, from among the plurality of indications of the products, at least one indication of a product containing a feature including at least part of the design by comparing the shape descriptors for the design to the shape descriptors corresponding to the at least one indication of the product; and
output, to the client device and via the network, a search result including the at least one product.
2. The system of claim 1 , wherein the instructions are further executable by the one or more processors to receive, as part of the search input, an indication of a region of the input image that includes the design.
3. The system of claim 2 , wherein the instructions are further executable by the one or more processors to generate a grayscale image for the region, and wherein to generate the edge image for the design is based on the grayscale image.
4. The system of claim 1 , wherein to generate the edge image for the design includes to fragment the input image into a plurality of blocks and perform an edge detection technique on each respective block of the plurality of blocks.
5. The system of claim 1 , wherein to compare the shape descriptors is based in part on a threshold amount of distortion between the shape descriptors.
6. The system of claim 5 , wherein the instructions are further executable by the one or more processors to receive, as part of the search input, a value for the threshold amount of distortion.
7. The system of claim 1 , wherein the instructions are further executable by the one or more processors to receive an input to filter the search result.
8. A method comprising:
generating, for each respective one of a plurality of indications of products, an edge image corresponding to an image associated with each one of the plurality of the indications of the products using a machine learning model to detect a feature depicted in the image, the edge image including lines identifying edges of the feature;
generating shape descriptors describing one or more pixel configurations or vector configurations for each respective one of the edge images;
receiving, from a client device and via a network, an input image including a design as part of a search input;
generating an edge image for the design based on the input image using the machine learning model;
generating shape descriptors for the design based on the edge image for the design;
locating, from among the plurality of indications of the products, at least one indication of a product containing a feature including at least part of the design by comparing the shape descriptors for the design to the shape descriptors corresponding to the at least one indication of the product; and
outputting, to the client device and via the network, a search result including the at least one product.
9. The method of claim 8 , further comprising receiving, as part of the search input, an indication of a region of the input image that includes the design.
10. The method of claim 9 , further comprising generating a grayscale image for the region, and wherein the generating the edge image for the design is based on the grayscale image.
11. The method of claim 8 , wherein the generating the edge image for the design includes fragmenting the input image into a plurality of blocks and performing an edge detection technique on each respective block of the plurality of blocks.
12. The method of claim 8 , wherein the comparing the shape descriptors is based in part on a threshold amount of distortion between the shape descriptors.
13. The method of claim 12 , further comprising receiving, as part of the search input, a value for the threshold amount of distortion.
14. The method of claim 8 , further comprising receiving an input to filter the search result.
15. A system comprising:
means for generating, for each respective one of a plurality of indications of products, an edge image corresponding to an image associated with each one of the plurality of the indications of the products using a machine learning model to detect a feature depicted in the image, the edge image including lines identifying edges of the feature;
means for generating shape descriptors describing one or more pixel configurations or vector configurations for each respective one of the edge images;
means for receiving, from a client device and via a network, an input image including a design as part of a search input;
means for generating an edge image for the design based on the input image using the machine learning model;
means for generating shape descriptors for the design based on the edge image for the design;
means for locating, from among the plurality of indications of the products, at least one indication of a product containing a feature including at least part of the design by comparing the shape descriptors for the design to the shape descriptors corresponding to the at least one indication of the product; and
means for outputting, to the client device and via the network, a search result including the at least one product.
16. The system of claim 15 , further comprising means for receiving, as part of the search input, an indication of a region of the input image that includes the design.
17. The system of claim 16 , further comprising means for generating a grayscale image for the region, and wherein the generating the edge image for the design is based on the grayscale image.
18. The system of claim 15 , wherein the generating the edge image for the design includes fragmenting the input image into a plurality of blocks and performing an edge detection technique on each respective block of the plurality of blocks.
19. The system of claim 15 , wherein the comparing the shape descriptors is based in part on a threshold amount of distortion between the shape descriptors.
20. The system of claim 19 , further comprising means for receiving, as part of the search input, a value for the threshold amount of distortion.