Granted Patent B2

US 9,928,476 · App. 13/597,734 · Granted Mar 27, 2018

Abstract

Claims (38)

1. A system for transforming a database of products, the system comprising:

a first data storage medium storing a raw product database that describes a plurality of products, wherein the raw product database includes, for each of the plurality of products, attributes associated with the products, wherein each attribute has a value that includes a word;

one or more processors;

a second data storage medium bearing instructions that, when executed on the one or more processors, cause the one or more processors to process the raw product database to provide a standardized data structure that contains standardized values for the attributes of the products for improved analytics by performing the steps of:

identify a dictionary comprising each word that appears as a value of at least one attribute of at least one product in the raw product database;

partition the dictionary into synonym classes of words, each synonym class forming an equivalence class of mutually equivalent terms for a product attribute, wherein when one of the values appears in two or more synonym classes, the value is stored with a context for differentiating between the two or more synonym classes;

select a representative word from each of the synonym classes, thereby associating each word in the dictionary with a corresponding representative word;

identify a collection of replacement rules, each of which replaces a target word from one of the synonym classes with the corresponding representative word, wherein the replacement rules comprise global rules, multi-word phrase rules, attribute constant rules, and single-word replacement rules; and

for each product in the raw product database, apply the replacement rules to one or more attributes of the product, thereby producing a standardized product database, wherein the global rules are applied before the multi-word phrase rules, the multi-word phrase rules are applied before the attribute constant rules, and the attribute constant rules are applied before the single-word replacement rules, wherein the replacement rules include at least one replacement rule for numerical values that is applied sequentially and repeatedly until a predetermined stopping condition is reached.

2. The system of claim 1 , further comprising instructions that cause the one or more processors to:

use the replacement rules, replacing each word in the dictionary with its corresponding representative, thereby producing a standardized dictionary;

add a new record to the standardized product database corresponding to a new product, wherein the new record includes attribute fields; and

select values for the attribute fields from the standardized dictionary.

3. The system of claim 1 , further comprising instructions that cause the one or more processors to:

define synonym classes of phrases in the dictionary.

4. The system of claim 1 , in which the rules are applied repeatedly until a pre-determined condition is met.

5. The system of claim 4 , in which the pre-determined condition includes the conditions that:

each rule is applied at once, and

there occurs an iteration of applying each of the rules such that the rules collectively fail to change at least one attribute value.

6. The system of claim 4 , in which the pre-determined condition includes application of the rules a pre-determined number of times.

7. The system of claim 2 , in which the new record is added based on detecting a new product in external data selected from the group consisting of: sales data from a retail outlet, inventory data from a retail outlet, a new product announcement from a media outlet, and a communication from a product manufacturer or affiliate.

8. The system of claim 2 , in which adding the new record includes:

automatically identifying a preliminary record based on raw external data;

applying the replacement rules to the preliminary record, thereby producing a transformed record;

submitting the transformed record to a human for review;

in response to human input, discarding the transformed record and receiving a new replacement rule from the human; and

applying the new replacement rule to the preliminary record, thereby producing the new record.

9. The system of claim 1 wherein the context includes one or more other values for one or more other attributes of the one of the plurality of products associated with the value of the attribute.

10. The system of claim 1 wherein the multi-word phrase rules are applied in increasing order of length of phrase.

11. The system of claim 1 wherein the multi-word phrase rules are applied in decreasing order of length of phrase.

12. The system of claim 1 wherein at least one of the synonym classes includes one or more common misspellings of a word.

13. The system of claim 1 wherein at least one of the synonym classes depends on context that determines a scope of a replacement rule for the one of the synonym classes.

14. The system of claim 2 further comprising a camera for acquiring a product image for the new product.

15. The system of claim 14 wherein the new record and the product image are sent to a human for review before adding the new record to the standardized database.

16. The system of claim 14 wherein the product image is scraped to identify product information for use as product attributes in the standardized database.

17. The system of claim 16 wherein the product information includes text extracted from the product image using optical character recognition.

18. The system of claim 16 wherein the product information includes image features detected in the product image.

19. The system of claim 16 wherein the product information includes a brand name or a logo.

Assignments (13)