Ontology-based attribute extraction from product descriptions
View Patent ↗Systems and methods are disclosed herein for obtaining a structured listing of attributes and corresponding values based on an unstructured document, such as a product description in a product record. Putative values are identified in the document and corresponding candidate attributes are identified in a taxonomy. Attribute-value pairs are then evaluated with respect to a plurality of rules. Attribute-value pairs and outputs of the one or more rules are evaluated using a machine-learning algorithm, such as a decision tree, in order to determine which attribute-value pairs to retain. Retained attribute-value pairs are stored and used to respond to search queries and facilitate comparison of products. Attributes selected may also be used to update a product template.
1. A method for classifying, the method comprising:
identifying, by a computer system, a plurality of values in a document;
identifying, by the computer system, a plurality of candidate attributes for the document based on the plurality of values;
selecting, by the computer system, one or more selected attributes from the plurality of candidate attributes according to an application of an attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document; and
associating, by the computer system, the one or more selected attributes with the document,
wherein:
selecting the one or more selected attributes from the plurality of candidate attributes according to the application of the attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document further comprises:
applying a machine learning algorithm to the plurality of candidate attributes and the document; and
the machine learning algorithm is a decision tree algorithm.
2. The method of claim 1 , wherein associating the one or more selected attributes with the document comprises:
associating the one or more selected attributes with a category to which the document belongs.
3. The method of claim 1 , wherein selecting the one or more selected attributes from the plurality of candidate attributes according to the application of the attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document further comprises:
applying the attribute selection rule taking as an input a set of predicted product types for the document.
4. The method of claim 1 , wherein selecting the one or more selected attributes from the plurality of candidate attributes according to the application of the attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document further comprises:
applying the attribute selection rule taking as an input a proximity of one or more of the plurality of values to a textual representation of a corresponding attribute of the plurality of candidate attributes.
5. The method of claim 1 , wherein selecting the one or more selected attributes from the plurality of candidate attributes according to the application of the attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document further comprises:
applying a rule with respect to a location of a corresponding value of the plurality of values in the document.
6. The method of claim 1 , wherein selecting the one or more selected attributes from the plurality of candidate attributes according to the application of the attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document further comprises:
applying a rule with respect to a usage of a value of the plurality of values corresponding to an attribute of the plurality of candidate attributes.
7. The method of claim 1 , wherein:
the document is a product record for a product.
8. The method of claim 7 , wherein associating the one or more selected attributes with the document further comprises:
augmenting the product record from an unstructured product record to a product record with structured attribute-value data.
9. A method for classifying, the method comprising:
identifying, by a computer system, a plurality of values in a document;
identifying, by the computer system, a plurality of candidate attributes for the document based on the plurality of values;
selecting, by the computer system, one or more selected attributes from the plurality of candidate attributes according to an application of an attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document; and
associating, by the computer system, the one or more selected attributes with the document,
wherein selecting the one or more selected attributes from the plurality of candidate attributes according to the application of the attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document further comprises:
applying a rule with respect to a part of speech of a value of the plurality of values corresponding to an attribute of the plurality of candidate attributes.
10. A method for classifying, the method comprising:
identifying, by a computer system, a plurality of values in a document;
identifying, by the computer system, a plurality of candidate attributes for the document based on the plurality of values;
selecting, by the computer system, one or more selected attributes from the plurality of candidate attributes according to an application of an attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document; and
associating, by the computer system, the one or more selected attributes with the document,
wherein selecting the one or more selected attributes from the plurality of candidate attributes according to the application of the attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document further comprises:
applying a rule with respect to whether a value of the plurality of values corresponding to an attribute of the plurality of candidate attributes is included within a superstring in a title of a node of a taxonomy.
11. A system for classifying, the system comprising one or more processors and one or more memory devices operably coupled to the one or more processors, the one or more memory devices storing executable and operational code effective to cause the one or more processors to:
identify a plurality of values in a document;
identify a plurality of candidate attributes for the document based on the plurality of values;
select one or more selected attributes from the plurality of candidate attributes according to an application of an attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document; and
associate the one or more selected attributes with the document,
wherein:
the executable and operational code are further effective to cause the one or more processors to select the one or more selected attributes from the plurality of candidate attributes according to the application of the attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document by:
applying a machine learning algorithm to the plurality of candidate attributes and the document; and
the machine learning algorithm is a decision tree algorithm.
12. The system of claim 11 , wherein the executable and operational code are further effective to cause the one or more processors to associate the one or more selected attributes with the document by:
associating the one or more selected attributes with a category to which the document belongs.
13. The system of claim 11 , wherein the executable and operational code are further effective to cause the one or more processors to select the one or more selected attributes from the plurality of candidate attributes according to the application of the attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document by:
applying the attribute selection rule taking as an input a set of predicted product types for the document.
14. The system of claim 11 , wherein the executable and operational code are further effective to cause the one or more processors to select the one or more selected attributes from the plurality of candidate attributes according to the application of the attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document by:
applying the attribute selection rule taking as an input a proximity of one or more of the plurality of values to a textual representation of a corresponding attribute of the plurality of candidate attributes.
15. The system of claim 11 , wherein the executable and operational code are further effective to cause the one or more processors to select the one or more selected attributes from the plurality of candidate attributes according to the application of the attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document by:
applying a rule with respect to a location of a corresponding value of the plurality of values in the document.
16. The system of claim 11 , wherein the executable and operational code are further effective to cause the one or more processors to select the one or more selected attributes from the plurality of candidate attributes according to the application of the attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document by:
applying a rule with respect to a usage of a value of the plurality of values corresponding to an attribute of the plurality of candidate attributes.
17. The system of claim 11 , wherein:
the document is a product record for a product.
18. The system of claim 17 , wherein the executable and operational code are further effective to cause the one or more processors to associate the one or more selected attributes with the document by:
augmenting the product record from an unstructured product record to a product record with structured attribute-value data, the structured attribute-value data comprising the one or more selected attributes.
19. A system for classifying, the system comprising one or more processors and one or more memory devices operably coupled to the one or more processors, the one or more memory devices storing executable and operational code effective to cause the one or more processors to:
identify a plurality of values in a document;
identify a plurality of candidate attributes for the document based on the plurality of values;
select one or more selected attributes from the plurality of candidate attributes according to an application of an attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document; and
associate the one or more selected attributes with the document,
wherein the executable and operational code are further effective to cause the one or more processors to select the one or more selected attributes from the plurality of candidate attributes according to the application of the attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document by:
applying a rule with respect to a part of speech of a value of the plurality of values corresponding to an attribute of the plurality of candidate attributes.
20. A system for classifying, the system comprising one or more processors and one or more memory devices operably coupled to the one or more processors, the one or more memory devices storing executable and operational code effective to cause the one or more processors to:
identify a plurality of values in a document;
identify a plurality of candidate attributes for the document based on the plurality of values;
select one or more selected attributes from the plurality of candidate attributes according to an application of an attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document; and
associate the one or more selected attributes with the document,
wherein the executable and operational code are further effective to cause the one or more processors to select the one or more selected attributes from the plurality of candidate attributes according to the application of the attribute selection rule to the plurality of values, the plurality of candidate attributes, and the document by:
applying a rule with respect to whether a value of the plurality of values corresponding to an attribute of the plurality of candidate attributes is included within a superstring in a title of a node of a taxonomy.