IP Library Granted Patent US 11,403,682
Granted Patent B2
US 11,403,682 · App. 16/427,238 · Granted Aug 2, 2022

Methods and apparatus for anomaly detections

Inventors: Jagdish Ramakrishnan (Foster City, CA); Elham Shaabani (Redwood City, CA); Chao Li (San Carlos, CA); Matyas A. Sustik (San Francisco, CA)
Assignee: Walmart Apollo, LLC
G06Q30/0283G06K9/6257G06K9/6262G06N20/00G06Q10/06312G06Q20/201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,403,682
App. No.
16/427,238
Granted
Aug 2, 2022
Kind
B2
Abstract

This application relates to apparatus and methods for identifying anomalies within data, such as pricing data. In some examples, a computing device receives data updates and selects a machine learning model to apply to the data update. The computing device may train the machine learning model with features generated based on historical purchase order data. An anomaly score is generated based on application of the machine learning model. Based on the anomaly score, the data update is either allowed, or denied. In some examples, the computing device re-trains the machine learning model with detected anomalies. In some embodiments, the computing device prioritizes detected anomalies for further investigation. In some embodiments, the computing device identifies the cause of the anomalies by identifying at least one feature that is causing the anomaly.

Claims (97)

1. A system comprising:

a computing device configured to:

receive a plurality of features that include a cost feature;

generate a feature score for each of the plurality of features based on a log transformation of each of the plurality of features with respect to the cost feature, wherein at least a portion of the log transformations comprise a ratio between each respective feature of the plurality of features and the cost feature;

receive a request to update a value;

determine a machine learning model to apply to the value based on the received request, wherein the machine learning model is trained based on the generated feature scores;

generate an anomaly score based on application of the machine learning model to the value;

determine whether the request is an anomaly based on the anomaly score;

allow the update to the value if the request is determined not to be an anomaly; and

if the request is determined to be an anomaly:

deny the update to the value;

store anomaly data identifying the anomaly to a database; and

re-train the machine learning model based on the stored anomaly data.

2. The system of claim 1 , wherein determining whether the request is an anomaly based on the anomaly score comprises

determining whether the anomaly score is beyond a predetermined amount.

3. The system of claim 2 , wherein generating the anomaly score for the value is based on

A

i

(

x

i

)

=

Σ

i

A

L

(

x

i

-

μ

i

)

2

σ

i

2

.

4. The system of claim 1 , wherein the plurality of features comprise at least one of a price feature, a binary feature, a categorical feature, and a hierarchical feature.

5. The system of claim 1 , wherein the machine learning model is an unsupervised machine learning model.

6. The system of claim 1 , wherein the value is at least one of a price and a cost of an item.

7. The system of claim 1 , wherein denying the update to the value if the request is determined to be the anomaly comprises:

generating a block update signal identifying that the update to the value is not to be allowed; and

transmitting the block update signal to a pricing system.

8. The system of claim 1 , wherein the computing device is configured to:

determine the request is the anomaly;

determine an impact score for the anomaly based on at least one of an estimated profit loss and a forgone revenue; and

determine a prioritization of the anomaly based on the determined impact score.

9. The system of claim 1 , wherein determining whether the request is an anomaly based on the anomaly score comprises comparing the anomaly score to a threshold.

10. A method comprising:

receiving a plurality of features that include a cost feature;

generating a feature score for each of the plurality of features based on a log transformation of each of the plurality of features with respect to the cost feature, wherein at least a portion of the log transformations comprise a ratio between each respective feature of the plurality of features and the cost feature;

receiving a request to update a value;

determining a machine learning model to apply to the value based on the received request, wherein the machine learning model is trained based on the generated feature scores;

generating an anomaly score based on application of the machine learning model to the value;

determining whether the request is an anomaly based on the anomaly score;

allowing the update to the value if the request is determined not to be an anomaly; and

if the request is determined to be an anomaly:

denying the update to the value;

storing anomaly data identifying the anomaly to a database; and

re-training the machine learning model based on the stored anomaly data.

11. The method of claim 10 wherein determining whether the request is an anomaly based on the anomaly score comprises:

generating an anomaly score for the value; and

determining whether the anomaly score is beyond a predetermined amount.

12. The method of claim 10 wherein the plurality of features comprising at least one of a price feature, a binary feature, a categorical feature, and a hierarchical feature.

13. The method of claim 10 wherein denying the update to the value if the request is determined to be the anomaly comprises:

generating a block update signal identifying that the update to the value is not to be allowed; and

transmitting the block update signal to a pricing system.

14. The method of claim 10 further comprising:

determining the request is the anomaly;

determining an impact score for the anomaly based on at least one of an estimated profit loss and a forgone revenue; and

determining a prioritization of the anomaly based on the determined impact score.

15. The method of claim 10 , wherein determining whether the request is an anomaly based on the anomaly score comprises comparing the anomaly score to a threshold.

16. A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by at least one processor, cause a device to perform operations comprising:

receiving a plurality of features that include a cost feature;

generating a feature score for each of the plurality of features based on a log transformation of each of the plurality of features with respect to the cost feature, wherein at least a portion of the log transformations comprise a ratio between each respective feature of the plurality of features and the cost feature;

receiving a request to update a value;

determining a machine learning model to apply to the value based on the received request, wherein the machine learning model is trained based on the generated feature scores;

generating an anomaly score based on application of the machine learning model to the value;

determining whether the request is an anomaly based on the anomaly score;

allowing the update to the value if the request is determined not to be an anomaly; and

if the request is determined to be an anomaly:

denying the update to the value;

storing anomaly data identifying the anomaly to a database; and

re-training the machine learning model based on the stored anomaly data.

17. The non-transitory computer readable medium of claim 16 further comprising instructions stored thereon that, when executed by at least one processor, further cause the device to perform operations comprising

determining whether the anomaly score is beyond a predetermined amount.

18. The non-transitory computer readable medium of claim 16 further comprising instructions stored thereon that, when executed by at least one processor, further cause the device to perform operations comprising training the machine learning model based on at a plurality of features comprising at least one of a price feature, a binary feature, a categorical feature, and a hierarchical feature.

19. The non-transitory computer readable medium of claim 16 further comprising instructions stored thereon that, when executed by at least one processor, further cause the device to perform operations comprising:

determining the request is the anomaly;

determining an impact score for the anomaly based on at least one of an estimated profit loss and a forgone revenue; and

determining a prioritization of the anomaly based on the determined impact score.

20. The non-transitory computer readable medium of claim 16 , wherein determining whether the request is an anomaly based on the anomaly score comprises comparing the anomaly score to a threshold.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 19, 2019
From: RAMAKRISHNAN, JAGDISH; SHAABANI, ELHAM; LI, CHAO; SUSTIK, MATYAS A.
To: WALMART APOLLO, LLC
Reel/Frame 049519/0834 →
Continuity (1)
Related Publication 20200380570A1 · Dec 3, 2020