System and method for association extraction for surf-shopping
The present disclosure is directed to a computer system and method performed by a selectively programmed data processor for providing data to a Web page such that items are presented to the user in a way that imitates a real world shopping experience. Various aspects of the disclosed technology also relate to systems and methods for calculating product or category associations using associative relation extraction. Additional aspects of the disclosed technology relate to automatic topic discovery, and event and category matching.
1. A computer implemented method for determining associative relationships comprising:
applying a programmed controlled process to select a category comprising a plurality of similar products stored in a database, wherein each of the plurality of similar products has a stored description;
constructing a category document associated with the category, comprising:
collecting the descriptions of each of the products within the category;
deleting a description that is a duplicate of another description; and
retaining descriptions of the category relating to one or more other categories;
applying the category document to a topic model to determine topics;
selecting related categories for the category document;
assigning one or more of said related categories to the determined topics;
selecting one or more of said retained descriptions from the category document to illustrate the relationship between the category document and a second category document associated with one of the related categories;
wherein said selected descriptions of the category document comprise all sentences from the descriptions of the products in the category document which mention the name of the category associated with the second category document;
selecting an exemplary product based on the selected descriptions; and
building a topic page.
2. The computer implemented method of claim 1 , wherein said topic model applies a Latent Dirichlet Allocation algorithm.
3. The computer implemented method of claim 1 , wherein building said topic page comprises k-means clustering to group topics together according to distance in a product ontology tree.
4. The computer implemented method of claim 1 , wherein the number of topics determined is at least ten times less than the number of categories.
5. The computer implemented method of claim 1 , wherein selecting related categories comprises sorting potential related categories according to the number of sentences in the category document that either mention one of the related categories or originated from a description of a product in one of the related categories.
6. A system for determining associative relationships comprising: a data processor programmed to:
select a category comprising a plurality of similar products, wherein each of the plurality of similar products has a description;
construct a category document associated with the category, comprising:
collecting the descriptions of each of the products within the category;
deleting a description that is a duplicate of another description; and
retaining descriptions of the category relating to one or more other categories;
apply the category document to a topic model to determine topics;
select related categories for the category document;
assign one or more of said related categories to the determined topics;
select one or more of said retained descriptions from the category document to illustrate the relationship between the category document and a second category document associated with one of the related categories;
wherein said selected descriptions of the category document comprise all sentences from the descriptions of the products in the category document which mention the name of the category associated with the second category document;
select an exemplary product based on the selected descriptions; and
build a topic page.
7. The system of claim 6 , wherein said topic model uses a Latent Dirichlet Allocation algorithm (LDA).
8. The system of claim 6 , wherein said data processor is further programmed to: build said topic page using LDA results to select product categories which represent the topic; and group those categories together, using k-means clustering, according to distance in a product ontology tree.
9. The system of claim 6 , wherein the number of topics determined is at least ten times less than the number of categories.
10. The system of claim 6 , wherein selecting related categories comprises sorting potential related categories according to the number of sentences in the category document that either mention one of the related categories or originated from a description of a product in one of the related categories.
11. A non-transitory computer readable storage medium containing programming, that when executed on a data processor, causes the data processor to perform steps comprising:
selecting a category comprising a plurality of similar products, wherein each of the plurality of similar products has a description;
constructing a category document associated with the category, comprising:
collecting the descriptions of each of the products within the category;
deleting a description that is a duplicate of another description; and
retaining descriptions of the category relating to one or more other categories;
applying the category document to a topic model to determine topics;
selecting related categories for the category document;
assigning one or more of said related categories to the determined topics;
selecting one or more of said retained descriptions from the category document to illustrate the relationship between the category document and a second category document associated with one of the related categories;
wherein said selected descriptions of the category document comprise all sentences from the descriptions of the products in the category document which mention the name of the category associated with the second category document;
selecting an exemplary product based on the selected descriptions; and
building a topic page.
12. The non-transitory computer readable storage medium of claim 11 , wherein said topic model uses a Latent Dirichlet Allocation algorithm.
13. The non-transitory computer readable storage medium of claim 11 , wherein building said topic page comprises k-means clustering to group topics together according to distance in a product ontology tree.
14. The non-transitory computer readable storage medium of claim 11 , wherein the number of topics determined is at least ten times less than the number of categories.
15. The non-transitory computer readable storage medium of claim 11 , wherein selecting related categories comprises sorting potential related categories according to the number of sentences in the category document that either mention one of the related categories or originated from a description of a product in one of the related categories.