Granted Patent B2

US 12,380,342 · App. 16/986,342 · Granted Aug 5, 2025

Inventors: Michael Elton Nidd (Zürich, CH); Hagen Völzer (Zürich, CH); Ioana Giurgiu (Zürich, CH); Jinho Hwang (Ossining, NY); Larisa Shwartz (Greenwich, CT)

Assignee: International Business Machines Corporation

G06N5/04G06F16/2379G06N5/01G06N20/20

Abstract

Claims (79)

1. A computer-implemented method for dynamically updating a master ruleset used by an alert management system that matches alerts in a data processing system with automated processes (“automata”) for handling the alerts, the method comprising:

training, by one or more processors, a machine learning model to correlate the alerts with the automata using a training dataset comprising matched alerts successfully handled by the automata, wherein the automata includes archiving logs in response to capacity alerts and removing zombie processes, wherein the machine learning model comprises a tree-ensemble model;

applying, by the one or more processors, the machine learning model to correlate unmatched alerts with the automata, wherein the unmatched alerts were not matched to the automata by the master ruleset;

responsive to exceeding a number of unmatched alerts, analyzing, by the one or more processors, an operation of the machine learning model in relation to correlation of the unmatched alerts, further comprising:

generating, by the one or more processors, a new tree based on features of paths through trees in the tree-ensemble model via which the unmatched alerts were correlated with the automata, wherein the new tree defines the new ruleset; and

pruning, by the one or more processors, the paths in dependance on a number on unmatched alerts correlated with automata via respective paths while retaining the paths with a higher number of unmatched alerts;

based on the analyzing, defining, by the one or more processors, a new ruleset for matching the unmatched alerts with the automata;

outputting, by the one or more processors, the new ruleset in a graphical user interface for auditing of each rule in the new ruleset;

responsive to approval of an audited rule of the new ruleset, adding, by the one or more processors, the audited rule to the master ruleset; and

responsive to matching the automata, deploying, by one or more processors, the automata.

2. The computer-implemented method of claim 1 , further comprising:

pruning, by the one or more processors, the paths based on a confidence value calculated for each path, wherein the confidence value is an estimation of a correlation accuracy; and

generating, by the one or more processors, the new tree based on features of unpruned paths.

3. The computer-implemented method of claim 1 , further comprising:

generating, by the one or more processors, the new tree based on features of unpruned paths.

4. The computer-implemented method of claim 1 , further comprising:

pruning, by the one or more processors, the paths based on a confidence value calculated for each path and on a number of the unmatched alerts correlated with the automata via respective paths, wherein the confidence value is an estimation of a correlation accuracy; and

generating, by the one or more processors, the new tree based on features of unpruned paths.

5. The computer-implemented method of claim 1 , wherein the step of analyzing operation of the machine learning model further comprises:

selecting, by the one or more processors, features from the trees of the tree-ensemble model based on significance values, each significance value indicating importance of the associated feature for correlating alerts in the training dataset with one of the automata with which a respective unmatched alert was correlated; and

generating, by the one or more processors, the new tree based on the selected features.

6. The computer-implemented method of claim 5 , further comprising:

pruning, by the one or more processors, the selected features based on a number of the unmatched alerts correlated with the automata via the paths including each feature; and

generating, by the one or more processors, the new tree based on unpruned features.

7. The computer-implemented method of claim 5 , further comprising:

pruning, by the one or more processors, the selected features based on confidence values calculated for respective paths via which the unmatched alerts were correlated with the automata and which included a respective selected feature of the selected features, wherein the confidence value is an estimation of a correlation accuracy; and

generating, by the one or more processors, the new tree based on unpruned features.

8. The computer-implemented method of claim 5 , further comprising:

pruning, by the one or more processors, the selected features based on a number of the unmatched alerts correlated with the automata via the paths including each feature, and based on confidence values calculated for the respective paths, wherein the confidence value is an estimation of a correlation accuracy; and

generating, by the one or more processors, the new tree based on unpruned features.

9. The computer-implemented method of claim 1 , wherein the step of analyzing operation of the machine learning model further comprises:

responsive to applying the machine learning model to obtain a correlation result for an unmatched alert, using, by the one or more processors, a model-independent explanation technique to identify at least one element of the alert as an important element for the correlation result; and

generating, by the one or more processors, the new ruleset based on the at least one element of the unmatched alerts.

10. The computer-implemented method of claim 1 , wherein the step of analyzing operation of the machine learning model further comprises:

when applying the machine learning model to correlate each unmatched alert, calculating, by the one or more processors, an initial confidence value for the initial correlation result, wherein the initial confidence value is an estimation of a correlation accuracy;

for different elements of the alert, removing, by the one or more processors, that element and reapplying the machine learning model to the resulting alert to calculate a new confidence value for the initial correlation result;

identifying, by the one or more processors, at least one element of the alert as an important element for the initial correlation result based on a difference between the initial confidence value and the new confidence value for the different elements; and

generating, by the one or more processors, the new ruleset based on the at least one element of the unmatched alerts.

11. The computer-implemented method of claim 1 , wherein the machine learning model comprises a boosted random forests model.

12. A computer program product for dynamically updating a master ruleset used by an alert management system that matches alerts in a data processing system with automated processes (“automata”) for handling the alerts, the computer program product comprising:

one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising:

program instructions to train a machine learning model to correlate the alerts with the automata using a training dataset comprising matched alerts successfully handled by the automata, wherein the automata includes archiving logs in response to capacity alerts and removing zombie processes, wherein the machine learning model comprises a tree-ensemble model;

program instructions to apply the machine learning model to correlate unmatched alerts with the automata, wherein the unmatched alerts were not matched to the automata by the master ruleset;

program instructions to, responsive to exceeding a number of unmatched alerts, analyze an operation of the machine learning model in relation to correlation of the unmatched alerts, wherein the program instructions further comprise:

program instructions to generate a new tree based on features of paths through trees in the tree-ensemble model via which the unmatched alerts were correlated with the automata, wherein the new tree defines the new ruleset; and

program instructions to prune the paths in dependance on a number on unmatched alerts correlated with automata via respective paths while retaining the paths with a higher number of unmatched alerts;

based on the program instructions to analyze, program instructions to define a new ruleset for matching the unmatched alerts with the automata;

program instructions to output the new ruleset in a graphical user interface for auditing of each rule in the new ruleset;

responsive to approval of an audited rule of the new ruleset, program instructions to add the audited rule to the master ruleset; and

responsive to matching the automata, program instructions to deploy the automata.

13. The computer program product of claim 12 , further comprising:

program instructions to prune the paths based on a confidence value calculated for each path, wherein the confidence value is an estimation of a correlation accuracy; and

program instructions to generate the new tree based on features of unpruned paths.

14. The computer program product of claim 12 , further comprising:

program instructions to generate the new tree based on the selected features.

15. The computer program product of claim 14 , further comprising:

program instructions to prune the selected features based on a number of the unmatched alerts correlated with the automata via the paths including each feature and based on confidence values calculated for respective paths via which the unmatched alerts were correlated with the automata and which included a respective selected feature, wherein the confidence value is an estimation of a correlation accuracy; and

program instructions to generate the new tree based on unpruned features.

16. The computer program product of claim 12 , wherein the program instructions to analyze operation of the machine learning model further comprise:

responsive to applying the machine learning model to obtain a correlation result for a respective unmatched alert, program instructions to use a model-independent explanation technique to identify at least one element of the unmatched alert as an important element for the correlation result; and

program instructions to generate the new ruleset based on the at least one element of the unmatched alerts.

17. The computer program product of claim 12 , wherein the program instructions to analyze operation of the machine learning model further comprise:

when applying the machine learning model to correlate each unmatched alert, program instructions to calculate an initial confidence value for the initial correlation result, wherein the initial confidence value is an estimation of a correlation accuracy;

for different elements of the alert, program instructions to remove that element and reapply the machine learning model to the resulting alert to calculate a new confidence value for the initial correlation result;

program instructions to identify at least one element of the alert as an important element for the initial correlation result based on a difference between the initial confidence value and the new confidence value for the different elements; and

program instructions to generate the new ruleset based on the at least one element of the unmatched alerts.

18. An alert-management system for dynamically updating a master ruleset used by an alert management system that matches alerts in a data processing system with automated processes (“automata”) for handling the alerts, the alert-management system comprising:

one or more computer processors;

one or more computer readable storage media;

program instructions collectively stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, the stored program instructions comprising:

program instructions to train a machine learning model to correlate the alerts with the automata using a training dataset comprising matched alerts successfully handled by the automata, wherein the automata includes archiving logs in response to capacity alerts and removing zombie processes, wherein the machine learning model comprises a tree-ensemble model;

program instructions to apply the machine learning model to correlate unmatched alerts with automata, wherein unmatched alerts were not matched to the automata by the master ruleset;

program instructions to, responsive to exceeding a number of unmatched alerts, analyze an operation of the machine learning model in relation to correlation of the unmatched alerts, wherein the program instructions further comprise:

program instructions to generate a new tree based on features of paths through trees in the tree-ensemble model via which the unmatched alerts were correlated with the automata, wherein the new tree defines the new ruleset; and

program instructions to prune the paths in dependance on a number on unmatched alerts correlated with automata via respective paths while retaining the paths with a higher number of unmatched alerts;

based on the program instructions to analyze, program instructions to define a new ruleset for matching the unmatched alerts with the automata;

program instructions to output the new ruleset in a graphical user interface for auditing of each rule in the new ruleset;

responsive to approval of an audited rule, program instructions to add the audited rule to the master ruleset; and

responsive to matching the automata, program instructions to deploy the automata.

Assignments (1)

Continuity (1)

References Cited (46)