Computer Vision and Image Characteristic Search
Computer vision and image characteristic search is described. The described system leverages visual search techniques by determining visual characteristics of objects depicted in images and comparing the determined characteristics to visual characteristics of other images, e.g., to identify similar visual characteristics in the other images. In some aspects, the described system performs searches that leverage a digital image as part of a search query to locate digital content of interest. In some aspects, the described system surfaces multiple user interface instrumentalities that include images of patterns, textures, or materials and that are selectable to initiate a visual search of digital content having a similar pattern, texture, or material. The described aspects also include pattern-based authentication in which the system determines authenticity of an item in an image based on a similarity of its visual characteristics to visual characteristics of known authentic items.
1 . A method implemented by at least one computing device, the method comprising:
monitoring, by the at least one computing device, user interaction with a user interface that outputs a live feed of digital images;
detecting, by the at least one computing device, user selection of at least one of the digital images;
inferring, by the at least one computing device, a characteristic of the selected digital image through comparison with at least one other digital image of the live feed;
generating, by the at least one computing device, a search query based at least in part on the inferred characteristic; and
outputting, by the at least one computing device, a search result in the user interface based on the search query.
2 . A method as described in claim 1 , further comprising:
detecting user selection of at least a second of the digital images;
inferring a different characteristic of the second digital image, the different inferred characteristic being different from the inferred characteristic; and
generating the search query based on both the inferred characteristic and the different inferred characteristic.
3 . A method as described in claim 2 , wherein the search result includes digital content having a first characteristic that is similar to the inferred characteristic and a second characteristic that is similar to the different inferred characteristic.
4 . A method as described in claim 1 , wherein inferring the characteristic is based on an amount of zoom of the selected digital image and the at least one other digital image.
5 . A method as described in claim 1 , wherein inferring the characteristic is based in part on whether the selected digital image depicts an entirety of an object or a limited portion of the object.
6 . A method as described in claim 1 , wherein the inferred characteristic is one of a pattern, a material, or a texture of an object partially depicted in the digital image.
7 . A method as described in claim 1 , wherein the inferred characteristic is a shape of an object entirely depicted in the digital image.
8 . A method as described in claim 1 , further comprising outputting an instrumentality via the user interface that is selectable to purchase an object depicted in a digital image of the search result.
9 . A method implemented by at least one computing device, the method comprising:
presenting, by the at least one computing device, a plurality of images via a user interface, each of the images depicting a different pattern;
receiving, by the at least one computing device, a selection of one of the images;
transmitting, by the at least one computing device, a search query including the selected image to a listing service configured to generate data describing a respective pattern of the selected image and identify listed items having a pattern that matches the respective pattern based on the generated data;
receiving, by the at least one computing device, search results that include at least one identified item having the matching pattern; and
presenting, by the at least one computing device, digital visual content depicting the at least one identified item via the user interface.
10 . A method as described in claim 9 , wherein:
the generated data is non-textual data describing the respective pattern; and
the identifying includes comparing the generated data to additional non-textual data describing patterns of the listed items as depicted in respective digital visual content.
11 . A method as described in claim 10 , wherein the non-textual data and the additional non-textual data comprise feature vectors.
12 . A method as described in claim 9 , wherein the presented digital visual content is a listing of the at least one identified item that enables purchase of the at least one identified item via the user interface.
13 . A method as described in claim 9 , wherein the search query is used to generate analytics data describing patterns searched for by users.
14 . A method as described in claim 9 , wherein the search query and the search results are used to generate analytics data describing search queries for the listed items having the pattern and availability of the listed items having the pattern.
15 . A method as described in claim 9 , wherein the matching pattern of the at least one identified item is similar to the respective pattern of the selected image.
16 . A method as described in claim 9 , wherein the matching pattern of the at least one identified item is the same as the respective pattern of the selected image.
17 . A method as described in claim 9 , wherein the plurality of images further include at least two images depicting different textures or different materials.
18 - 20 . (canceled)
21 . A system comprising:
one or more processors; and
memory having stored thereon computer-readable instructions that are executable by the one or more processors to perform operations comprising:
presenting a plurality of images via a user interface, each of the images depicting a different pattern;
receiving a selection of one of the images via the user interface;
transmitting a search query including the selected image to a listing service configured to generate data describing a respective pattern of the selected image and identify listed items having a pattern that matches the respective pattern based on the generated data;
receiving search results that include at least one identified item having the matching pattern; and
presenting digital visual content depicting the at least one identified item via the user interface.
22 . A system as described in claim 21 , wherein:
the generated data is non-textual data describing the respective pattern; and
the identifying includes comparing the generated data to additional non-textual data describing patterns of the listed items as depicted in respective digital visual content.
23 . A system as described in claim 22 , wherein the non-textual data and the additional non-textual data comprise feature vectors.