IP Library › Granted Patent US 11,151,467
Granted Patent B1
US 11,151,467 · App. 15/807,348 · Granted Oct 19, 2021

System, method, and computer program for generating intelligent automated adaptive decisions

Inventors: Moshe Yechiel Shtein (Hod Hasharon, IL); Gilad Barkan (Kfar Monash, IL); Pinchas Faran (Kadima-Tzoran, IL)
Assignee: AMDOCS DEVELOPMENT LIMITED
G06N7/00G06N20/00G06Q30/0204
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,151,467
App. No.
15/807,348
Granted
Oct 19, 2021
Kind
B1
Abstract

A system, method, and computer program product are provided for generating intelligent automated adaptive decisions. In operation, a system receives a request to generate a prediction associated with a business problem for a customer. The system identifies a customer segment from a plurality of customer segments to which the new customer is most closely associated. Additionally, the system identifies a statistical model associated with the customer segment. Moreover, the system selects a best prediction from a competing set of machine learning models and the statistical model, utilizing a multi arm bandit arbitrator applying a multi arm bandit technique to solve the business problem.

Claims (38)

1. A method, comprising:

generating, by a system, a plurality of customer segments by clustering an entire population of customers into distinct and mutual exclusive clusters utilizing a clustering model;

receiving, by the system, a request to generate a prediction associated with a new business problem for a new customer;

selecting, by the system, a customer segment from the plurality of customer segments to which the new customer is most closely associated;

identifying, by the system, a statistical model associated with the selected customer segment;

obtaining, by the system, a prediction of a solution to the new business problem output by the statistical model;

obtaining, by the system, a plurality of additional predictions of solutions to the new business problem, each additional prediction of the plurality of additional predictions output by a different machine learning model in a set of competing machine learning models; and

applying, by the system, a multi arm bandit technique of a multi arm bandit arbitrator to the prediction output by the statistical model and the plurality of additional predictions output by the set of competing machine learning models, to select a best prediction from among the prediction output by the statistical model and the plurality of additional predictions output by the set of competing machine learning models.

2. The method of claim 1 , wherein the plurality of customer segments are periodically generated.

3. The method of claim 1 , wherein one statistical model is generated per cluster by applying the multi arm bandit technique during a cold start period associated with the system.

4. The method of claim 3 , wherein results of applying the multi arm bandit technique during the cold start period associated with the system are stored as training observations for training a plurality of machine learning models including the set of competing machine learning models.

5. The method of claim 4 , wherein the system determines when enough of the training observations and the plurality of machine learning models exist to exit the cold start period and selects the best prediction of a best performing model from the set of competing machine learning models and the statistical model, utilizing the multi arm bandit arbitrator applying the multi arm bandit technique to solve the business problem.

6. The method of claim 5 , wherein a business outcome as a result of the best performing model is utilized to train the plurality of machine learning models and the statistical model.

7. A computer program product embodied on a non-transitory computer readable medium, comprising computer code for:

generating, by a system, a plurality of customer segments by clustering an entire population of customers into distinct and mutual exclusive clusters utilizing a clustering model;

receiving, by the system, a request to generate a prediction associated with a new business problem for a new customer;

selecting, by the system, a customer segment from the plurality of customer segments to which the new customer is most closely associated;

identifying, by the system, a statistical model associated with the selected customer segment;

obtaining, by the system, a prediction of a solution to the new business problem output by the statistical model;

obtaining, by the system, a plurality of additional predictions of solutions to the new business problem, each additional prediction of the plurality of additional predictions output by a different machine learning model in a set of competing machine learning models; and

applying, by the system, a multi arm bandit technique of a multi arm bandit arbitrator to the prediction output by the statistical model and the plurality of additional predictions output by the set of competing machine learning models, to select a best prediction from among the prediction output by the statistical model and the plurality of additional predictions output by the set of competing machine learning models.

8. The computer program product of claim 7 , wherein the plurality of customer segments are periodically generated.

9. The computer program product of claim 7 , wherein one statistical model is generated per cluster by applying the multi arm bandit technique during a cold start period associated with the system.

10. The computer program product of claim 9 , wherein results of applying the multi arm bandit technique during the cold start period associated with the system are stored as training observations for training a plurality of machine learning models including the set of competing machine learning models.

11. The computer program product of claim 10 , wherein the system determines when enough of the training observations and the plurality of machine learning models exist to exit the cold start period and selects the best prediction of a best performing model from the set of competing machine learning models and the statistical model, utilizing the multi arm bandit arbitrator applying the multi arm bandit technique to solve the business problem.

12. The computer program product of claim 11 , wherein a business outcome as a result of the best performing model is utilized to train the plurality of machine learning models and the statistical model.

13. A system, comprising one or more processors, operable for:

generating, by the system, a plurality of customer segments by clustering an entire population of customers into distinct and mutual exclusive clusters utilizing a clustering model;

receiving, by the system, a request to generate a prediction associated with a new business problem for a new customer;

selecting, by the system, a customer segment from the plurality of customer segments to which the new customer is most closely associated;

identifying, by the system, a statistical model associated with the selected customer segment;

obtaining, by the system, a prediction of a solution to the new business problem output by the statistical model;

obtaining, by the system, a plurality of additional predictions of solutions to the new business problem, each additional prediction of the plurality of additional predictions output by a different machine learning model in a set of competing machine learning models; and

applying, by the system, a multi arm bandit technique of a multi arm bandit arbitrator to the prediction output by the statistical model and the plurality of additional predictions output by the set of competing machine learning models, to select a best prediction from among the prediction output by the statistical model and the plurality of additional predictions output by the set of competing machine learning models.

14. The system of claim 13 , wherein the plurality of customer segments are periodically generated.

15. The system of claim 13 , wherein one statistical model is generated per cluster by applying the multi arm bandit technique during a cold start period associated with the system.

16. The system of claim 15 , wherein results of applying the multi arm bandit technique during the cold start period associated with the system are stored as training observations for training a plurality of machine learning models including the set of competing machine learning models.

17. The system of claim 16 , wherein the system determines when enough of the training observations and the plurality of machine learning models exist to exit the cold start period and selects the best prediction of a best performing model from the set of competing machine learning models and the statistical model, utilizing the multi arm bandit arbitrator applying the multi arm bandit technique to solve the business problem.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 8, 2018
From: SHTEIN, MOSHE YECHIEL; BARKAN, GILAD; FARAN, PINCHAS
To: AMDOCS DEVELOPMENT LIMITED
Reel/Frame 044563/0249 →
Cited By (3)
US 12,608,640 US 12,689,868 US 12,749,003