Interactive clothes searching in online stores
A clothing search system provides a clothing search to users using a component-based image search. Retailer catalogs are analyzed to determine clothing components within clothing images. Features associated with the components are determined. When a user requests a clothing search, the clothing search system selects clothing based on the components and features requested by the user. The user may also provide an image to the clothing search system. The clothing search system determines components and features of the image and identifies clothing with matching components.
1. A method for interactively searching clothes in online stores, the method comprising:
training a clothing search system to detect components of articles of clothing and features of the components from photos, the training comprising:
maintaining templates based on the photos, each template corresponding to a component and structured as a tree, a branch of the tree corresponding to a set of common characteristics of the component across the photos;
receiving, by the clothing search system, a plurality of photos of a plurality of articles of clothing sold by a seller;
for each photo of an article of clothing:
automatically detecting, by the clothing search system, a plurality of components associated with the article of clothing by applying the templates to the photo, and
automatically extracting, by the clothing search system from the photo, a plurality of features corresponding to the plurality of components;
receiving, by the clothing search system, a query from a user for an article of clothing sold by the seller; and
retrieving, by the clothing search system based on the query, the article of clothing, the retrieving comprising:
identifying a component of the article of clothing from the query and matching the component to an automatically detected component, and
identifying from the query a feature that corresponds to the component and matching the feature to an automatically extracted feature.
2. The method of claim 1 , further comprising building a database for storing the plurality of photos of articles of clothing, the automatically detected plurality of components, and the automatically extracted plurality of features.
3. The method of claim 1 , wherein the automatically extracted plurality of features comprise:
type of the article of clothing;
color of the article of clothing;
style of the article of clothing;
location of each component of the article of clothing;
description of each component; and
design information of the article of clothing.
4. The method of claim 1 , wherein the templates comprise shape templates of individual human body parts.
5. The method of claim 4 , wherein training the tree-structure human parsing model comprises:
training the shape templates of individual human body parts based on a large amount of pre-labeled training samples for human parsing; and
training tree-structured combinations of human body parts subject to a large amount of pre-labeled training samples for human parsing.
6. The method of claim 1 , wherein retrieving the article of clothing comprises:
determining relevancy of the matched component and the matched feature.
7. The method of claim 1 , further comprising:
annotating common clothing styles and storing the annotation with the articles of clothing;
evaluating the plurality of components and the plurality of features using the annotations; and
searching for common clothing styles of among the plurality of articles of clothing stored using the annotations.
8. A non-transitory computer-readable storage medium storing executable computer program instructions for interactively searching clothes in online stores, the computer program instructions comprising instructions for:
training a clothing search system to detect components of articles of clothing and features of the components from photos, the training comprising:
maintaining templates based on the photos, each template corresponding to a component and structured as a tree, a branch of the tree corresponding to a set of common characteristics of the component across the photos;
receiving, by the clothing search system, a plurality of photos of a plurality of articles of clothing sold by a seller;
for each photo of an article of clothing:
automatically detecting, by the clothing search system, a plurality of components associated with the article of clothing by applying the templates to the photo, and
automatically extracting, by the clothing search system from the photo, a plurality of features corresponding to the plurality of components;
receiving, by the clothing search system, a query from a user for an article of clothing sold by the seller; and
retrieving, by the clothing search system based on the query, the article of clothing, the retrieving comprising:
identifying a component of the article of clothing from the query and matching the component to an automatically detected component, and
identifying from the query a feature that corresponds to the component and matching the feature to an automatically extracted feature.