IP Library Granted Patent US 11,645,575
Granted Patent B2
US 11,645,575 · App. 16/239,276 · Granted May 9, 2023

Linking actions to machine learning prediction explanations

Inventors: Elizabeth Daly (Dublin, IE); Oznur Alkan (Clonsilla, IE); Massimiliano Mattetti (Dublin, IE); Inge Vejsbjerg (Kilmainham, IE)
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
G06N20/00G06N5/02
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,645,575
App. No.
16/239,276
Granted
May 9, 2023
Kind
B2
Abstract

Embodiments for recommending actions to improve machine learning predictions by a processor. One or more recommended actions may be linked to one or more features that influence a predicted outcome of a prediction model of a machine learning operation. One or more features having one or more negative factors that negatively impact the predicted outcome of the prediction model may be determined and selected. One or more of the linked recommended actions may be applied to one or more of the features to mitigate a negative impact upon the predicted outcome of the prediction model.

Claims (56)

1. A method for recommending actions to improve machine learning predictions by a processor, comprising:

receiving training data to build a predictive model;

executing machine learning logic to generate the predictive model using the training data;

analyzing the predictive model to determine one or more features that influence a predicted outcome of the prediction model, wherein the analyzing includes selecting those of a plurality of features used in the predictive model being associated with an action having a least monetary cost, as compared with alternative actions being associated with alternative features, which improve the predicted outcome as the one or more features;

identifying one or more recommended actions linked to the one or more features that, when performed, modify the one or more features that influence the predicted outcome of the prediction model, wherein the one or more recommended actions are generated specifically for the predictive model only after a first instance of an execution of the predictive model, and wherein the identifying of the one or more recommended actions includes:

receiving, on a user interface (UI), opportunity data associated with a transactional operation between entities,

predicting, by the predictive model according to an analyzation of the opportunity data, an opportunities success rate for the transactional operation,

inputting the opportunities success rate and a set of confidence scores to a model explainer,

identifying from the input, by the model explainer, the one or more features having one or more negative factors that negatively impact the predicted outcome of the predictive model,

querying, by the model explainer, one or more recommender engines to identify the one or more recommended actions, wherein the one or more recommender engines infer whether the one or more recommended actions are able to be executed internally by the processor, and

transmitting the one or more recommended actions to a service orchestrator, wherein the service orchestrator issues one or more service call operations to trigger the one or more recommended actions to be performed; and

executing the one or more recommended actions to modify the one or more features, wherein the executing of the one or more recommended actions results in an increase in an accuracy of the predicted outcome of the prediction model.

2. The method of claim 1 , further including determining those of the one or more features having the one or more negative factors that negatively impact the predicted outcome of the prediction model.

3. The method of claim 1 , further including applying the recommended actions to the one or more features to mitigate a negative impact upon the predicted outcome of the prediction model.

4. The method of claim 1 , further including:

ranking those of the one or more features having the one or more negative factors that negatively impact the predicted outcome of the prediction model; and

applying the recommended actions to one or more ranked features to mitigate a negative impact upon the predicted outcome of the prediction model caused by the one or more negative factors.

5. The method of claim 1 , further including creating one or more links between one or more services that provide the one or more recommended actions.

6. A system for recommending actions to improve machine learning predictions, comprising:

one or more computers with executable instructions that when executed cause the system to:

receive training data to build a predictive model;

execute machine learning logic to generate the predictive model using the training data;

analyze the predictive model to determine one or more features that influence a predicted outcome of the prediction model, wherein the analyzing includes selecting those of a plurality of features used in the predictive model being associated with an action having a least monetary cost, as compared with alternative actions being associated with alternative features, which improve the predicted outcome as the one or more features;

identify one or more recommended actions linked to the one or more features that, when performed, modify the one or more features that influence the predicted outcome of the prediction model, wherein the one or more recommended actions are generated specifically for the predictive model only after a first instance of an execution of the predictive model, and wherein the identifying of the one or more recommended actions includes:

receiving, on a user interface (UI), opportunity data associated with a transactional operation between entities,

predicting, by the predictive model according to an analyzation of the opportunity data, an opportunities success rate for the transactional operation,

inputting the opportunities success rate and a set of confidence scores to a model explainer,

identifying from the input, by the model explainer, the one or more features having one or more negative factors that negatively impact the predicted outcome of the predictive model,

querying, by the model explainer, one or more recommender engines to identify the one or more recommended actions, wherein the one or more recommender engines infer whether the one or more recommended actions are able to be executed internally by the processor, and

transmitting the one or more recommended actions to a service orchestrator, wherein the service orchestrator issues one or more service call operations to trigger the one or more recommended actions to be performed; and

execute the one or more recommended actions to modify the one or more features, wherein the executing of the one or more recommended actions results in an increase in an accuracy of the predicted outcome of the prediction model.

7. The system of claim 6 , wherein the executable instructions further determine those of the one or more features having the one or more negative factors that negatively impact the predicted outcome of the prediction model.

8. The system of claim 6 , wherein the executable instructions further apply the recommended actions to the one or more features to mitigate a negative impact upon the predicted outcome of the prediction model.

9. The system of claim 6 , wherein the executable instructions further:

rank those of the one or more features having the one or more negative factors that negatively impact the predicted outcome of the prediction model; and

apply the recommended actions to one or more ranked features to mitigate a negative impact upon the predicted outcome of the prediction model caused by the one or more negative factors.

10. The system of claim 6 , wherein the executable instructions further create one or more links between one or more services that provide the one or more recommended actions.

11. A computer program product for recommending actions to improve machine learning predictions by a processor, the computer program product comprising a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

an executable portion that receives training data to build a predictive model;

an executable portion that executes machine learning logic to generate the predictive model using the training data;

an executable portion that analyzes the predictive model to determine one or more features that influence a predicted outcome of the prediction model, wherein the analyzing includes selecting those of a plurality of features used in the predictive model being associated with an action having a least monetary cost, as compared with alternative actions being associated with alternative features, which improve the predicted outcome as the one or more features;

an executable portion that identifies one or more recommended actions linked to the one or more features that, when performed, modify the one or more features that influence the predicted outcome of the prediction model, wherein the one or more recommended actions are generated specifically for the predictive model only after a first instance of an execution of the predictive model, and wherein the identifying of the one or more recommended actions includes:

receiving, on a user interface (UI), opportunity data associated with a transactional operation between entities,

predicting, by the predictive model according to an analyzation of the opportunity data, an opportunities success rate for the transactional operation,

inputting the opportunities success rate and a set of confidence scores to a model explainer,

identifying from the input, by the model explainer, the one or more features having one or more negative factors that negatively impact the predicted outcome of the predictive model,

querying, by the model explainer, one or more recommender engines to identify the one or more recommended actions, wherein the one or more recommender engines infer whether the one or more recommended actions are able to be executed internally by the processor, and

transmitting the one or more recommended actions to a service orchestrator, wherein the service orchestrator issues one or more service call operations to trigger the one or more recommended actions to be performed; and

an executable portion that executes the one or more recommended actions to modify the one or more features, wherein the executing of the one or more recommended actions results in an increase in an accuracy of the predicted outcome of the prediction model.

12. The computer program product of claim 11 , further including an executable portion that:

determines those of the one or more features having the one or more negative factors that negatively impact the predicted outcome of the prediction model; and

applies the recommended actions to the one or more features to mitigate a negative impact upon the predicted outcome of the prediction model.

13. The computer program product of claim 11 , further including an executable portion that:

ranks those of the one or more features having the one or more negative factors that negatively impact the predicted outcome of the prediction model; and

applies the recommended actions to one or more ranked features to mitigate a negative impact upon the predicted outcome of the prediction model caused by the one or more negative factors.

14. The computer program product of claim 11 , further including an executable portion that creates one or more links between one or more services that provide the one or more recommended actions.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 23, 2019
From: DALY, ELIZABETH; ALKAN, OZNUR; MATTETTI, MASSIMILIANO; VEJSBJERG, INGE
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 049829/0551 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 3, 2019
From: DALY, ELIZABETH; ALKAN, OZNUR; MATTETTI, MASSIMILIANO; VEJSBJERG, INGE
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 047895/0401 →
Continuity (1)
Related Publication 20200219004A1 · Jul 9, 2020