Automated correction of attributes using machine-learned large language models (LLMS)
An online system maintains a product catalog including products from various retailers, from which users can purchase products. Each of the products are associated with attributes such as a size value and a size unit of measure (UOM). The online system identifies products with erroneous product attributes using taxonomy attribute value homogeneity. The online system performs an inference task in conjunction with the model serving system or the interface system to infer a correct size value and size UOM of the product. The online system evaluates the accuracy of the inferred size value and size UOM of the product. Responsive to determining that the inferred data is accurate, the online system updates the product catalog with the corrected product attribute information.
1 . A method comprising:
accessing a catalog database of items, wherein each item of the catalog of products is associated with a set of attributes, the set of attributes comprising at least a size value and a size unit of measure;
identifying an item determined to include one or more candidate errors in the set of attributes for the item;
identifying an expected size value and an expected unit of measure for the item;
generating a prompt for input to a machine-learned language model, the prompt specifying at least item data, one or more size values and size unit of measures of the item from one or more sources, and a request to infer a correct size value and size unit of measure of the item;
providing the prompt to a model serving system deploying the machine-learned language model for execution by the machine-learned language model;
receiving, from the model serving system, a response generated by executing the machine-learned language model on the prompt;
parsing the response from the model serving system to extract a size value and a size unit of measure for the item; updating the catalog database with the extracted size value and the extracted size unit of
measure in association with the item, wherein updating the catalog database comprises:
comparing the extracted size value and the extracted unit of measure from the response to the expected size value and the expected unit of measure; and
responsive to identifying that the extracted size value aligns with the expected size value and the extracted unit of measure aligns with the expected unit of measure, updating the catalog database with the extracted size value and extracted unit of measure as the size value and size unit of measure of the product.
2 . The method of claim 1 , further comprising:
responsive to identifying that the extracted size value does not align with the expected size value and the extracted unit of measure does not align with the expected unit of measure, providing the extracted size value and the extracted unit of measure to quality audit for validation.
3 . The method of claim 1 , wherein identifying the expected size value and the expected unit of measure further comprises:
identifying a taxonomy node for the item within a taxonomy structure;
identifying size values and unit of measures for a plurality of items within the taxonomy node; and
setting the expected size value and the expected unit of measure for the item as a size value and a unit of measure identified for a threshold number or proportion of items within the taxonomy node.
4 . The method of claim 1 , further comprising:
obtaining a set of estimated size values and unit of measures for the item including the extracted size value and the extracted unit of measure from the machine-learned language model;
assigning a priority score to each of the set of estimated size values and unit of measures;
ranking the priority scores; and
selecting the extracted size value and the extracted unit of measure having a threshold priority score as a correct size of value and unit of measure for the item.
5 . The method of claim 1 , further comprising:
validating the response generated by executing the machine-learned language model on the prompt, the validating comprising: identifying a difference between the response and an expected size value and an expected unit of measure for the item;
generating feedback reflecting accuracy of the response based on the validation; and
updating the prompt to the machine-learned language model using the feedback.
6 . The method of claim 5 , further comprising: updating the machine-learned language model using the feedback, wherein updating the machine-learned language model comprises:
generating one or more training examples using the feedback, each training example comprising an item that is validated, and validated size value and validated size unit of measure for the item;
applying the machine-learned language model to the one or more training examples;
obtaining an output size value and an output size unit of measure for the item from the machine-learned language model;
generating a loss that indicates a difference between the output size value and the output size of measure and the validated size value and validated size unit of measure; and
backpropagating terms obtained from the loss to update parameters of the machine-learned language model.
7 . The method of claim 1 , further comprising:
retrieving the size value and the unit of measure for the item from the catalog database;
generating a content item including the retrieved size value and the unit of measure; and
providing the content item for display on a client device.
8 . A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to perform steps comprising:
accessing a catalog database of items, wherein each item of the catalog of products is associated with a set of attributes, the set of attributes comprising at least a size value and a size unit of measure;
identifying an item determined to include one or more candidate errors in the set of attributes for the item;
identifying an expected size value and an expected unit of measure for the item;
generating a prompt for input to a machine-learned language model, the prompt specifying at least item data, one or more size values and size unit of measures of the item from one or more sources, and a request to infer a correct size value and size unit of measure of the item;
providing the prompt to a model serving system deploying the machine-learned language model for execution by the machine-learned language model;
receiving, from the model serving system, a response generated by executing the machine-learned language model on the prompt;
parsing the response from the model serving system to extract a size value and a size unit of measure for the item;
updating the catalog database with the extracted size value and the extracted size unit of measure in association with the item, wherein updating the catalog database comprises:
comparing the extracted size value and the extracted unit of measure from the response to the expected size value and the expected unit of measure; and
responsive to identifying that the extracted size value aligns with the expected size value and the extracted unit of measure aligns with the expected unit of measure, updating the catalog database with the extracted size value and extracted unit of measure as the size value and size unit of measure of the product.
9 . The computer program product of claim 8 , wherein the instructions, when executed by a processor, cause the processor to perform steps comprising:
responsive to identifying that the extracted size value does not align with the expected size value and the extracted unit of measure does not align with the expected unit of measure, providing the extracted size value and the extracted unit of measure to quality audit for validation.
10 . The computer program product of claim 8 , wherein identifying the expected size value and the expected unit of measure further comprises:
identifying a taxonomy node for the item within a taxonomy structure;
identifying size values and unit of measures for a plurality of items within the taxonomy node; and
setting the expected size value and the expected unit of measure for the item as a size value and a unit of measure identified for a threshold number or proportion of items within the taxonomy node.
11 . The computer program product of claim 8 , wherein the instructions, when executed by a processor, cause the processor to perform steps comprising:
obtaining a set of estimated size values and unit of measures for the item including the extracted size value and the extracted unit of measure from the machine-learned language model;
assigning a priority score to each of the set of estimated size values and unit of measures;
ranking the priority scores; and
selecting the extracted size value and the extracted unit of measure having a threshold priority score as a correct size of value and unit of measure for the item.
12 . The computer program product of claim 8 , wherein the instructions, when executed by a processor, cause the processor to perform steps comprising:
validating the response generated by executing the machine-learned language model on the prompt, the validating comprising: identifying a difference between the response and an expected size value and an expected unit of measure for the item;
generating feedback reflecting accuracy of the response based on the validation; and
updating the prompt to the machine-learned language model using the feedback.
13 . The computer program product of claim 12 , wherein the instructions, when executed by a processor, cause the processor to perform steps comprising:
updating the machine-learned language model using the feedback, wherein updating the machine-learned language model comprises:
generating one or more training examples using the feedback, each training example comprising an item that is validated, and validated size value and validated size unit of measure for the item;
applying the machine-learned language model to the one or more training examples;
obtaining an output size value and an output size unit of measure for the item from the machine-learned language model;
generating a loss that indicates a difference between the output size value and the output size of measure and the validated size value and validated size unit of measure; and
backpropagating terms obtained from the loss to update parameters of the machine-learned language model.
14 . The computer program product of claim 8 , wherein the instructions, when executed by a processor, cause the processor to perform steps comprising:
retrieving the size value and the unit of measure for the item from the catalog database;
generating a content item including the retrieved size value and the unit of measure; and
providing the content item for display on a client device.
15 . A computer system comprising:
a processor; and
a non-transitory computer-readable storage medium having instructions that, when executed by the processor, cause the computer system to perform steps comprising:
accessing a catalog database of items, wherein each item of the catalog of products is associated with a set of attributes, the set of attributes comprising at least a size value and a size unit of measure;
identifying an item determined to include one or more candidate errors in the set of attributes for the item;
identifying an expected size value and an expected unit of measure for the item;
generating a prompt for input to a machine-learned language model, the prompt specifying at least item data, one or more size values and size unit of measures of the item from one or more sources, and a request to infer a correct size value and size unit of measure of the item;
providing the prompt to a model serving system deploying the machine-learned language model for execution by the machine-learned language model;
receiving, from the model serving system, a response generated by executing the machine-learned language model on the prompt;
parsing the response from the model serving system to extract a size value and a size unit of measure for the item;
updating the catalog database with the extracted size value and the extracted size unit of measure in association with the item, wherein updating the catalog database comprises:
comparing the extracted size value and the extracted unit of measure from the response to the expected size value and the expected unit of measure; and
responsive to identifying that the extracted size value aligns with the expected size value and the extracted unit of measure aligns with the expected unit of measure, updating the catalog database with the extracted size value and extracted unit of measure as the size value and size unit of measure of the product.
16 . The system of claim 15 , wherein instructions that, when executed by the processor, cause the computer system to perform steps comprising:
obtaining a set of estimated size values and unit of measures for the item including the extracted size value and the extracted unit of measure from the machine-learned language model;
assigning a priority score to each of the set of estimated size values and unit of measures;
ranking the priority scores; and
selecting the extracted size value and the extracted unit of measure having a threshold priority score as a correct size of value and unit of measure for the item.
17 . The system of claim 15 , wherein instructions that, when executed by the processor, cause the computer system to perform steps comprising:
validating the response generated by executing the machine-learned language model on the prompt, the validating comprising: identifying a difference between the response and an expected size value and an expected unit of measure for the item;
generating feedback reflecting accuracy of the response based on the validation; and
updating the prompt to the machine-learned language model using the feedback.