Granted Patent B2

US 12,333,561 · App. 17/840,279 · Granted Jun 17, 2025

Inventors: Hamed Janani (Vancouver, CA); Anirudh Challa (Burnaby, CA); Hong Wang (San Mateo, CA); Mohamad Al-Sharara (Westmount, CA); José R. Benkí (Ann Arbor, MI); Zealand Cooley (Albion, MI)

Assignee: Verint Americas Inc.

G06Q30/0203G06F18/214G06Q30/0201

Abstract

Claims (94)

1. A method for detecting anomalies in actual survey score results, the method comprising:

receiving, by a preprocessing component, historic survey score data from a survey scores database;

generating, by the preprocessing component, a set of training survey score data for a given scored survey metric and a given duration from the historic survey score data, using a score formatting component, a grouping component, and a division component;

generating, by a modeling component, a survey score prediction model for the given scored survey metric and the given duration;

training the survey score prediction model to generate an expected survey score for the given scored survey metric and given duration with the set of training survey score data;

receiving, by the preprocessing component, an actual survey score result associated with a scored survey metric and a duration for a specific date;

determining, by the preprocessing component, the scored survey metric and the duration for the actual survey score result match the given scored survey metric and the given duration for the survey score prediction model;

passing, to the modeling component, a current survey score data associated with the actual survey score result to use the survey score prediction model associated with the given scored survey metric and the given duration;

generating, by the modeling component, an expected survey score for the specific date based on the current survey score data using the survey score prediction model;

receiving, by an anomaly detection component, a user-defined confidence level and a user-defined standard deviation augmentation;

generating, by the anomaly detection component, a confidence interval for the expected survey score based on the user-defined confidence level;

generating, by the anomaly detection component, a standard deviation band for the expected survey score based on a standard deviation and the user-defined standard deviation augmentation;

determining, by the anomaly detection component, the actual survey score result is an anomaly when the actual survey score result is outside the confidence interval and when the actual survey score result is outside of the standard deviation band;

generating, by the anomaly detection component, an anomaly report for the actual survey score result;

updating, by the modeling component, the survey score prediction model with a set of prediction survey score data to improve the survey score prediction model's ability to predict the expected survey score, wherein the set of survey score prediction data includes the current survey score data; and

replacing the survey score prediction model with the updated survey score prediction model.

2. The method of claim 1 , wherein the survey score prediction model is a deep learning model or a statistical model.

3. The method of claim 1 , further comprising determining the actual survey score result is not an anomaly when the actual survey score result is within the confidence interval or when the actual survey score result is within the standard deviation band.

4. The method of claim 1 , further comprising:

determining at least one of the scored survey metric or the duration for the actual survey score result does not match the given scored survey metric or the given duration for the survey score prediction model;

searching a model store database for a different survey score prediction model for the scored survey metric and the duration associated with the actual survey score result;

receiving the different survey score prediction model for the scored survey metric and the duration associated with the actual survey score result from the model store database;

passing the current survey score data associated with the actual survey score result to the different survey score prediction model; and

generating the expected survey score for the specific date using the different survey score prediction model.

5. The method of claim 4 , further comprising:

determining the different survey score prediction model for the scored survey metric and the duration associated with the actual survey score result is not in the model store database;

generating a different set of training survey score data for the scored survey metric and the duration associated with the actual survey score result from the historic survey score data;

generating, by the modeling component, the different survey score prediction model for the scored survey metric and the duration associated with the actual survey score result; and

training the different survey score prediction model for the scored survey metric and the duration associated with the actual survey score result with the different set of training survey score data.

6. The method of claim 1 , further comprising tuning a set of hyperparameters for the survey score prediction model with a set of testing survey score data and selecting a model variation from a plurality of model variations that has a lowest error score for use as the survey score prediction model.

7. The method of claim 1 , further comprising storing the survey score prediction model in a model store database.

8. A system for detecting anomalies in actual survey score results, the system comprising:

a memory comprising computer readable instructions;

a processor configured to read the computer readable instructions that when executed causes the system to:

receive historic survey score data from a survey scores database;

generate a set of training survey score data for a given scored survey metric and a given duration from the historic survey score data;

generate a survey score prediction model for the given survey metric and the given duration;

train the survey score prediction model to generate an expected survey score for the given scored survey metric and given duration with the set of training survey score data;

receive an actual survey score result associated with a scored survey metric and a duration for a specific date;

determine the scored survey metric and the duration for the actual survey score result match the given scored survey metric and the given duration for the survey score prediction model;

pass a current survey score data associated with the actual survey score result to use the survey score prediction model associated with the given scored survey metric and the given duration;

generate an expected survey score for the specific date based on the current survey score data using the survey score prediction model;

receive a user-defined confidence level and a user-defined standard deviation augmentation;

generate a confidence interval for the expected survey score based on the user-defined confidence level;

generate a standard deviation band for the expected survey score based on a standard deviation and the user-defined standard deviation augmentation;

determine the actual survey score result is an anomaly when the actual survey score result is outside the confidence interval and when the actual survey score result is outside of the standard deviation band;

generate an anomaly report for the actual survey score result

update the survey score prediction model with a set of prediction survey score data to improve the survey score prediction model's ability to predict the expected survey score, wherein the set of survey score prediction data includes the current survey score data; and

replace the survey score prediction model with the updated survey score prediction model.

9. The system of claim 8 , wherein the survey score prediction model is a deep learning model or a statistical model.

10. The system of claim 8 , wherein the system is further caused to determine the actual survey score result is not an anomaly when the actual survey score result is within the confidence interval or when the actual survey score result is within the standard deviation band.

11. The system of claim 8 , wherein the system is further caused to:

determine at least one of the scored survey metric or the duration for the actual survey score result does not match the given scored survey metric or the given duration for the survey score prediction model;

search a model store database for a different survey score prediction model for the scored survey metric and the duration associated with the actual survey score result;

receive the different survey score prediction model for the scored survey metric and the duration associated with the actual survey score result from the model store database;

pass the current survey score data associated with the actual survey score result to the different survey score prediction model; and

generate the expected survey score for the specific date using the different survey score prediction model.

12. The system of claim 11 , wherein the system is further caused to:

determine the different survey score prediction model for the scored survey metric and the duration associated with the actual survey score result is not in the model store database;

generate a different set of training survey score data for the scored survey metric and the duration associated with the actual survey score result from the historic survey score data;

generate the different survey score prediction model for the scored survey metric and the duration associated with the actual survey score result; and

train the different survey score prediction model for the scored survey metric and the duration associated with the actual survey score result with the different set of training survey score data.

13. The system of claim 8 , wherein the system is further caused to tune a set of hyperparameters for the survey score prediction model with a set of testing survey score data and selecting a model variation from a plurality of model variations that has a lowest error score for use as the survey score prediction model.

14. The system of claim 8 , wherein the system is further caused to store the survey score prediction model in a model store database.

15. A non-transitory computer readable medium comprising computer readable code to detect anomalies in actual survey score results on a system that when executed by a processor, causes the system to:

receive historic survey score data from a survey scores database;

generate a set of training survey score data for a given scored survey metric and a given duration from the historic survey score data;

generate a survey score prediction model for the given scored survey metric and the given duration;

train the survey score prediction model to generate an expected survey score for the given scored survey metric and given duration with the set of training survey score data;

receive an actual survey score result associated with a scored survey metric and a duration for a specific date;

determine the scored survey metric and the duration for the actual survey score result match the given scored survey metric and the given duration for the survey score prediction model;

pass a current survey score data associated with the actual survey score result to use the survey score prediction model associated with the given scored survey metric and given duration;

generate an expected survey score for the specific date based on the current survey score data using the survey score prediction model;

receive a user-defined confidence level and a user-defined standard deviation augmentation;

generate a confidence interval for the expected survey score based on the user-defined confidence level;

generate a standard deviation band for the expected survey score based on a standard deviation and the user-defined standard deviation augmentation;

determine the actual survey score result is an anomaly when the actual survey score result is outside the confidence interval and when the actual survey score result is outside of the standard deviation band;

generate a anomaly report for the actual survey score result

update, by the modeling component, the survey score prediction model with a set of prediction survey score data to improve the survey score prediction model's ability to predict the expected survey score, wherein the set of survey score prediction data includes the current survey score data; and

replace the survey score prediction model with the updated survey score prediction model.

16. The non-transitory computer readable medium of claim 15 , wherein the survey score prediction model is a recurrent neural network based model, a transformer based model, or an autoregressive based statistical model.

17. The non-transitory computer readable medium of claim 15 , wherein the system is further caused to:

determine at least one of the scored survey metric or the duration for the actual survey score result does not match the given scored survey metric or the given duration for the survey score prediction model;

search a model store database for a different survey score prediction model for the scored survey metric and the duration associated with the actual survey score result;

receive the different survey score prediction model for the scored survey metric and the duration associated with the actual survey score result from the model store database;

pass the current survey score data associated with the actual survey score result to the different survey score prediction model; and

generate the expected survey score for the specific date using the different survey score prediction model.

18. The non-transitory computer readable medium of claim 17 , wherein the system is further caused to:

determine the different survey score prediction model for the scored survey metric and the duration associated with the actual survey score result is not in the model store database;

generate a different set of training survey score data for the scored survey metric and the duration associated with the actual survey score result from the historic survey score data;

generate the different survey scored prediction model for the scored survey metric and the duration associated with the actual survey score result; and

train the different survey score prediction model for the scored survey metric and the duration associated with the actual survey score result with the different set of training survey score data.

19. The non-transitory computer readable medium of claim 15 , wherein the system is further caused to tune a set of hyperparameters for the survey score prediction model with a set of testing survey score data and selecting a model variation from a plurality of model variations that has a lowest error score for use as the survey score prediction model.

20. The non-transitory computer readable medium of claim 15 , wherein the system is further caused to determine the actual survey score result is not an anomaly when the actual survey score result is within the confidence interval or when the actual survey score result is within the standard deviation band.

Assignments (3)

Continuity (1)

References Cited (10)