Granted Patent B2

US 12,744,801 · App. 18/159,703 · Granted Sep 22, 2026

Inventors: Zhi Li (Beijing, CN); Xiao Dong Li (Beijing, CN); He Jiang Jia (Beijing, CN); Xing Xing Shen (Beijing, CN); Ye Tian (Dalian, CN); Sheng Jie Han (Beijing, CN)

Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION

H04L63/1425H04L41/16

Abstract

Claims (60)

1 . A computer-implemented method of facilitating processing within a computing environment, the computer-implemented method comprising:

obtaining system metrics data generated during executing one or more test cases configured to test functionality within a system;

using a machine learning implicit-anomaly model to determine a possibility of an implicit anomaly associated with executing the one or more test cases within the system;

determining, using artificial intelligence, an occurrence of an incident within the system based on detecting an anomaly that is not indicated by a result of executing the one or more test cases within the system, wherein detecting the anomaly uses the determined possibility of the implicit anomaly within the system based on the system metrics data; and

initiating an action based on the occurrence of the incident within the system associated with detecting the anomaly not indicated by the result, and despite the one or more test cases completing successfully.

2 . The computer-implemented method of claim 1 , further comprising:

training the machine learning implicit-anomaly model using training system metrics data collected from one or more test systems executing the one or more test cases.

3 . The computer-implemented method of claim 1 , wherein obtaining of the system metrics data comprises:

generating performance-robustness data vectors from collected performance data of the system.

4 . The computer-implemented method of claim 3 , wherein obtaining the system metrics data further comprises:

clustering system text data logs into functional areas; and

classifying the generated performance-robustness data vectors using the functional areas to obtain the system metrics data referenced by the machine learning implicit-anomaly model.

5 . The computer-implemented method of claim 1 , further comprising:

using a machine learning explicit-anomaly model to determine a possibility of an explicit anomaly within the system based on system text data obtained from executing the one or more test cases on the system; and

wherein determining the occurrence of the incident associated with executing the one or more test cases further comprises using the determined possibility of the explicit anomaly within the system based on the system text data in addition to the determined possibility of the implicit anomaly within the system based on the system metrics data, wherein the system metrics data and the system text data are different types of system data collected from executing the one or more test cases on the system.

6 . The computer-implemented method of claim 5 , wherein obtaining of the system metrics data comprises:

generating performance-robustness data vectors from collected performance data of the system;

clustering system text data logs into functional areas; and

classifying the generated performance-robustness data vectors using the functional areas to obtain the system metrics data referenced by the machine learning implicit-anomaly model.

7 . The computer-implemented method of claim 6 , wherein obtaining of the system text data comprises:

collecting and parsing system text output obtained from executing the one or more test cases on the system; and

converting the collected and parsed system text output into text data vectors, the system text data comprising the text data vectors.

8 . The computer-implemented method of claim 5 , wherein the artificial intelligence comprises a multilayer machine learning model, and wherein determining the occurrence of the incident comprises using the multilayer machine learning model in determining the occurrence of the incident within the system based on the determined possibility of implicit anomaly within the system using the system metrics data, and the determined possibility of explicit anomaly within the system using the system text data.

9 . The computer-implemented method of claim 1 , wherein the system is a production system and initiating the action comprises modifying operation of the system based on determining the occurrence of the incident within the system associated with executing the one or more test cases.

10 . A computer system for facilitating processing within a computing environment, the computer system comprising:

a memory; and

at least one processor in communication with the memory, wherein the computer system is configured to perform a method, the method comprising:

obtaining system metrics data generated during executing one or more test cases configured to test functionality within a system;

using a machine learning implicit-anomaly model to determine a possibility of an implicit anomaly associated with executing the one or more test cases within the system;

determining, using artificial intelligence, an occurrence of an incident within the system based on detecting an anomaly that is not indicated by a result of executing the one or more test cases within the system, wherein detecting the anomaly uses the determined possibility of the implicit anomaly within the system based on the system metrics data; and

initiating an action based on the occurrence of the incident within the system associated with detecting the anomaly not indicated by the result, and despite the one or more test cases completing successfully.

11 . The computer system of claim 10 , further comprising:

training the machine learning implicit-anomaly model using training system metrics data collected from one or more test systems executing the one or more test cases.

12 . The computer system of claim 10 , wherein obtaining of the system metrics data comprises:

generating performance-robustness data vectors from collected performance data of the system;

clustering system text data logs into functional areas; and

classifying the generated performance-robustness data vectors using the functional areas to obtain the system metrics data referenced by the machine learning implicit-anomaly model.

13 . The computer system of claim 10 , further comprising:

using a machine learning explicit-anomaly model to determine a possibility of an explicit anomaly within the system based on system text data obtained from executing the one or more test cases on the system; and

wherein determining the occurrence of the incident associated with executing the one or more test cases further comprises using the determined possibility of the explicit anomaly within the system based on the system text data in addition to the determined possibility of the implicit anomaly within the system based on the system metrics data, wherein the system metrics data and the system text data are different types of system data collected from executing the one or more test cases on the system.

14 . The computer system of claim 13 , wherein obtaining of the system metrics data comprises:

generating performance-robustness data vectors from collected performance data of the system;

clustering system text data logs into functional areas; and

classifying the generated performance-robustness data vectors using the functional areas to obtain the system metrics data referenced by the machine learning implicit-anomaly model.

15 . The computer system of claim 14 , wherein obtaining of the system text data comprises:

collecting and parsing system text output obtained from executing the one or more test cases on the system; and

converting the collected and parsed system text output into text data vectors, the system text data comprising the text data vectors.

16 . The computer system of claim 13 , wherein the artificial intelligence comprises a multilayer machine learning model, and wherein determining the occurrence of the incident comprises using the multilayer machine learning model in determining the occurrence of the incident within the system based on the determined possibility of implicit anomaly within the system using the system metrics data, and the determined possibility of explicit anomaly within the system using the system text data.

17 . A computer program product for facilitating processing within a computing environment, the computer program product comprising:

one or more computer-readable storage media and program instructions collectively stored on the one or more computer-readable storage media readable by at least one processing circuit to cause the processing circuit to perform a method comprising:

obtaining system metrics data generated during executing one or more test cases configured to test functionality within a system;

using a machine learning implicit-anomaly model to determine a possibility of an implicit anomaly associated with executing the one or more test cases within the system;

determining, using artificial intelligence, an occurrence of an incident within the system based on detecting an anomaly that is not indicated by a result of executing the one or more test cases within the system, wherein detecting the anomaly uses the determined possibility of the implicit anomaly within the system based on the system metrics data; and

initiating an action based on the occurrence of the incident within the system associated with detecting the anomaly not indicated by the result, and despite the one or more test cases completing successfully.

18 . The computer program product of claim 17 , further comprising:

training the machine learning implicit-anomaly model using training system metrics data collected from one or more test systems executing the one or more test cases.

19 . The computer program product of claim 17 , further comprising:

using a machine learning explicit-anomaly model to determine a possibility of an explicit anomaly within the system based on system text data obtained from executing the one or more test cases on the system; and

wherein determining the occurrence of the incident associated with executing the one or more test cases further comprises using the determined possibility of the explicit anomaly within the system based on the system text data in addition to the determined possibility of the implicit anomaly within the system based on the system metrics data, wherein the system metrics data and the system text data are different types of system data collected from executing the one or more test cases on the system.

20 . The computer program product of claim 19 , wherein the artificial intelligence comprises a multilayer machine learning model, and wherein determining the occurrence of the incident comprises using the multilayer machine learning model in determining the occurrence of the incident within the system based on the determined possibility of implicit anomaly within the system using the system metrics data, and the determined possibility of explicit anomaly within the system using the system text data.

Assignments (1)

Continuity (1)

References Cited (18)