Granted Patent B2

US 11,640,388 · App. 17/245,063 · Granted May 2, 2023

Inventors: Xiang Yu Yang (Xian, CN); Deng Xin Luo (Xian, CN); Ye Wang (Xian, CN); Yu Pan (Shanghai, CN); Zhong Fang Yuan (Xian, CN); Miao Guo (Beijing, CN)

Assignee: International Business Machines Corporation

G06F16/2365G06F16/288G06F16/9024G06F18/2433G06F18/24147

Abstract

Claims (84)

1. A computer-implemented method comprising:

obtaining pre-check data associated with data nodes, wherein the pre-check data includes data associated with a production environment;

generating a machine learning data graph model based on abstracting data and data flows of the production environment, wherein each node of a plurality of nodes of the data graph model represents different data and an edge of the data graph model represents a relationship between nodes of the data graph;

determining a value for each edge of the data graph model based on an attenuation factor;

determining one or more specified nodes from the plurality of nodes of the data graph model;

determining initial hyperparameters for the data graph model based at least on historical data associated with the production environment represented by the data graph model, wherein the hyperparameters include the attenuation factor;

calculating outliers for each specified data node by an outlier detection algorithm, wherein the outliers are calculated based on a unit of the pre-check data associated with each specified data node;

backtracking the calculated outliers for each specified data node through an associated generating data link;

generating preliminary high-probability anomaly nodes as a set of largest outliers based on calculating outliers for the one or more specified nodes and backtracking the calculated outliers along input edges;

determining one or more of the preliminary high-probability anomaly nodes are true anomaly nodes;

generating an evaluation of the abstracted structure represented by the data graph model;

selecting one or more data nodes and one or more data links associated with the set of largest outliers; and

generating potential anomaly indications based on the one or more data nodes selected and the one or more data links selected.

2. The computer-implemented method of claim 1 , wherein backtracking the calculated outliers for each specified data node through the associated generating data link comprises:

backward propagating the outliers along an input edge representing the generating data link of each specified data node;

applying an attenuation factor to each outlier propagation along an edge; and

adding the outliers on any intersection nodes.

3. The computer-implemented method of claim 2 , wherein the attenuation factor for the edge is based on a distance from an outlier node.

4. The computer-implemented method of claim 1 , further comprising:

determining the abstracted structure represented by the data graph model meets defined requirements based on the evaluation; and

providing the initial hyperparameters as cured hyperparameters for the data graph.

5. The computer-implemented method of claim 1 , further comprising:

determining the abstracted structure represented by the data graph model does not meet defined requirements based on the evaluation;

performing one or more training iterations to generate a next evaluation until a determination is made that requirements are met based on the next evaluation, the training iterations comprising:

generating adjusted hyperparameters;

generating a next iteration of high-probability anomaly nodes based on the adjusted hyperparameters; and

generating the next evaluation of the abstracted structure represented by the data graph model;

determining the abstracted structure represented by the data graph model meets defined requirements based on the next evaluation; and

providing the adjusted hyperparameters as cured hyperparameters for the data graph model.

6. The computer-implemented method of claim 1 , wherein an outlier calculation for a node is defined based on a type of data.

7. A computer program product comprising one or more computer readable storage media and collectively stored program instructions on the one or more computer readable storage media, the stored program instructions comprising:

program instructions programmed to obtain pre-check data associated with data nodes, wherein the pre-check data includes data associated with a production environment;

program instructions programmed to generate a machine learning data graph model based on abstracting data and data flows of the production environment, wherein each node of a plurality of nodes of the data graph model represents different data and an edge of the data graph model represents a relationship between nodes of the data graph;

program instructions programmed to determine a value for each edge of the data graph model based on an attenuation factor;

program instructions programmed to determine one or more specified nodes from the plurality of nodes of the data graph model;

program instructions programmed to determine initial hyperparameters for the data graph model based at least on historical data associated with the production environment represented by the data graph model, wherein the hyperparameters include the attenuation factor;

program instructions programmed to calculate outliers for each specified data node by an outlier detection algorithm, wherein the outliers are calculated based on a unit of the pre-check data associated with each specified data node;

program instructions programmed to backtrack the calculated outliers for each specified data node through an associated generating data link;

program instructions programmed to generate preliminary node data based on an abstracted structure represented by the data graph;

program instructions programmed to generate preliminary high-probability anomaly nodes as a set of largest outliers based on calculating outliers for the one or more specified nodes and backtracking the calculated preliminary outliers along input edges;

program instructions to determine one or more of the preliminary high-probability anomaly nodes are true anomaly nodes;

program instructions to generate an evaluation of the abstracted structure represented by the data graph model;

program instructions programmed to select one or more data nodes and one or more data links associated with the set of largest outliers; and

program instructions programmed to generate potential anomaly indications based on the one or more data nodes selected and the one or more data links selected.

8. The computer program product of claim 7 , wherein backtracking the calculated outliers for each specified data node through the associated generating data link comprises:

backward propagating the outliers along an input edge representing the generating data link of each specified data node;

applying an attenuation factor to each outlier propagation along an edge; and

adding the outliers on any intersection nodes.

9. The computer program product of claim 8 , wherein the attenuation factor for the edge is based on a distance from an outlier node.

10. The computer program product of claim 7 , the computer readable storage medium having further stored thereon:

program instructions programmed to determine the abstracted structure represented by the data graph model meets defined requirements based on the evaluation; and

program instructions programmed to provide the initial hyperparameters as cured hyperparameters for the data graph model.

11. The computer program product of claim 7 , the computer readable storage medium having further stored thereon:

program instructions programmed to determine the abstracted structure represented by the data graph model does not meet defined requirements based on the evaluation;

program instructions programmed to perform one or more training iterations to generate a next evaluation until a determination is made that requirements are met based on the next evaluation, the training iterations comprising:

program instructions programmed to generate adjusted hyperparameters;

program instructions programmed to generate a next iteration of high-probability anomaly nodes based on the adjusted hyperparameters; and

program instructions programmed to generate the next evaluation of the abstracted structure represented by the data graph model;

program instructions programmed to determine the abstracted structure represented by the data graph model meets defined requirements based on the next evaluation; and

program instructions programmed to provide the adjusted hyperparameters as cured hyperparameters for the data graph model.

12. The computer program product of claim 7 , wherein an outlier calculation for a node is defined based on a type of data.

13. A computer system comprising:

a processor set; and

a computer readable storage medium;

wherein:

the processor set is structured, located, connected and programmed to run program instructions stored on the computer readable storage medium; and

the stored program instructions include:

program instructions programmed to obtain pre-check data associated with data nodes, wherein the pre-check data includes data associated with a defined environment;

program instructions programmed to generate a machine learning data graph model based on abstracting data and data flows of the production environment, wherein each node of a plurality of nodes of the data graph model represents different data and an edge of the data graph model represents a relationship between nodes of the data graph;

program instructions programmed to determine a value for each edge of the data graph model based on an attenuation factor;

program instructions programmed to determine one or more specified nodes from the plurality of nodes of the data graph model;

program instructions programmed to determine initial hyperparameters for the data graph model based at least on historical data associated with the production environment represented by the data graph model, wherein the hyperparameters include the attenuation factor;

program instructions programmed to calculate outliers for each specified data node by an outlier detection algorithm, wherein the outliers are calculated based on a unit of the pre-check data associated with each specified data node;

program instructions programmed to backtrack the calculated outliers for each specified data node through an associated generating data link;

program instructions programmed to generate preliminary node data based on an abstracted structure represented by the data graph;

program instructions programmed to generate preliminary high-probability anomaly nodes as a set of largest outliers based on calculating outliers for the one or more specified nodes and backtracking the calculated preliminary outliers along input edges;

program instructions to determine one or more of the preliminary high-probability anomaly nodes are true anomaly nodes;

program instructions to generate an evaluation of the abstracted structure represented by the data graph model;

program instructions programmed to select one or more data noes and one or more data links associated with the set of largest outliers; and

program instructions programmed to generate potential anomaly indications based on the one or more data nodes selected and the one or more data links selected.

14. The computer system of claim 13 , wherein backtracking the calculated outliers for each specified data node through the associated generating data link comprises:

backward propagating the outliers along an input edge representing the generating data link of each specified data node;

applying an attenuation factor to each outlier propagation along an edge; and

adding the outliers on any intersection nodes.

Assignments (1)

Continuity (1)

Cited By (100)

US 12,206,696 US 12,244,621 US 12,261,866 US 12,267,345 US 12,284,197 US 12,309,181 US 12,309,182 US 12,309,185 US 12,309,236 US 12,323,449 US 12,335,286 US 12,335,348 US 12,341,797 US 12,348,545 US 12,355,626 US 12,355,787 US 12,355,793 US 12,363,148 US 12,368,745 US 12,368,746 US 12,368,747 US 12,375,573 US 12,381,901 US 12,395,573 US 12,401,669 US 12,405,849 US 12,407,701 US 12,407,702 US 12,418,552 US 12,418,555 US 12,425,428 US 12,425,430 US 12,445,474 US 12,452,272 US 12,452,279 US 12,457,231 US 12,463,994 US 12,463,995 US 12,463,996 US 12,463,997 US 12,464,003 US 12,470,577 US 12,470,578 US 12,483,576 US 12,489,770 US 12,489,771 US 12,495,052 US 12,500,910 US 12,500,911 US 12,500,912 US 12,505,126 US 12,506,762 US 12,511,110 US 12,513,221 US 12,526,297 US 12,537,836 US 12,537,837 US 12,537,839 US 12,537,840 US 12,537,884 US 12,549,575 US 12,549,577 US 12,556,548 US 12,556,559 US 12,563,060 US 12,563,064 US 12,563,071 US 12,563,072 US 12,580,932 US 12,580,934 US 12,580,935 US 12,580,936 US 12,580,937 US 12,587,553 US 12,592,950 US 12,598,205 US 12,613,930 US 12,615,271 US 12,621,324 US 12,621,329 US 12,627,686 US 12,627,687 US 12,627,690 US 12,634,312 US 12,634,376 US 12,652,302 US 12,659,325 US 12,659,326 US 12,659,327 US 12,659,333 US 12,676,874 US 12,689,638 US 12,689,640 US 12,695,768 US 12,706,931 US 12,706,932 US 12,706,933 US 12,706,980 US 12,712,897 US 12,719,896