IP Library › Granted Patent US 11,734,141
Granted Patent B2
US 11,734,141 · App. 17/375,078 · Granted Aug 22, 2023

Dynamic testing of systems

Inventors: David Verburg (Mantorville, MN); Gábor Andorkó (Budapest, HU); Oscar Alejandro de la Torre Del Rio (Zapopan, MX); Richard Burton Finch (New Paltz, NY)
Assignee: International Business Machines Corporation
G06F11/26G06F11/079G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,734,141
App. No.
17/375,078
Granted
Aug 22, 2023
Kind
B2
Abstract

Aspects of the invention include receiving system data associated with a first system, the first system comprising a plurality of system components, wherein the system data comprises component data for each system component in the plurality of system components, obtaining historical performance data for each system component in the plurality of system components, determining at least one testing constraint associated with the first system, determining a test environment for the first system, the test environment comprising a plurality of test cases for the first system based on the system data, the historical performance data, and the at least one testing constraint, and executing the test environment on the first system.

Claims (67)

1. A computer-implemented method comprising:

receiving system data associated with a first system, the first system comprising a plurality of system components, wherein the system data comprises component data for each system component in the plurality of system components;

obtaining historical performance data for each system component in the plurality of system components;

determining at least one testing constraint associated with the first system;

determining a test environment for the first system, the test environment comprising a plurality of test cases for the first system based on the system data, the historical performance data, and the at least one testing constraint; and

executing the test environment on the first system,

wherein determining the test environment for the first system comprises:

generating, via a machine learning model, a first feature vector comprising a plurality of features extracted from the component data;

plotting the first feature vector in a multi-variate feature space; and

determining the test environment based on a location of the first feature vector in the multi-variate feature space.

2. The computer-implemented method of claim 1 , further comprising:

determining a test time for test environment for the first system based at least in part on the system data, the historical performance data, and the at least one testing constraint.

3. The computer-implemented method of claim 1 , wherein the at least one testing constraint comprises a total testing time limit constraint, and the method further comprises:

determining an available time period based on the total testing time limit and the testing time for the testing environment; and

collecting learning data on the first system during the available time period.

4. The computer-implemented method of claim 3 , wherein collecting the learning data on the first system comprises:

executing a learning test case on the first system during the available time period; and

adjusting one or more system parameters of the first system during the learning test case execution.

5. The computer-implemented method of claim 4 , wherein the one or more system parameters comprise at least one of a voltage adjustment, a frequency adjustment, and a temperature adjustment.

6. The computer-implemented method of claim 1 , wherein the plurality of system components comprise one or more of a memory card, a cooling system, and a processor core.

7. The computer-implemented method of claim 1 , wherein determining the at least one testing constraint comprises:

determining a first system test budget;

calculating a testing time limit based on the first system test budget; and

determining the at least one testing constraint as the testing time limit.

8. A system comprising:

a memory having computer readable instructions; and

one or more processors for executing the computer readable instructions, the computer readable instructions controlling the one or more processors to perform operations comprising:

receiving system data associated with a first system, the first system comprising a plurality of system components, wherein the system data comprises component data for each system component in the plurality of system components;

obtaining historical performance data for each system component in the plurality of system components;

determining at least one testing constraint associated with the first system;

determining a test environment for the first system, the test environment comprising a plurality of test cases for the first system based on the system data, the historical performance data, and the at least one testing constraint; and

executing the test suite on the first system,

wherein determining the test environment for the first system comprises:

generating, via a machine learning model, a first feature vector comprising a plurality of features extracted from the component data;

plotting the first feature vector in a multi-variate feature space; and

determining the test environment based on a location of the first feature vector in the multi-variate feature space.

9. The system of claim 8 , wherein the operations further comprise:

determining a test time for test environment for the first system based at least in part on the system data, the historical performance data, and the at least one testing constraint.

10. The system of claim 8 , wherein the at least one testing constraint comprises a total testing time limit constraint, and the operations further comprise:

determining an available time period based on the total testing time limit and the testing time for the testing environment; and

collecting learning data on the first system during the available time period.

11. The system of claim 10 , wherein collecting the learning data on the first system comprises:

executing a learning test case on the first system during the available time period; and

adjusting one or more system parameters of the first system during the learning test case execution.

12. The system of claim 11 , wherein the one or more system parameters comprise at least one of a voltage adjustment, a frequency adjustment, and a temperature adjustment.

13. The system of claim 8 , wherein the plurality of system components comprise one or more of a memory card, a cooling system, and a processor core.

14. The system of claim 8 , wherein determining the at least one testing constraint comprises:

determining a first system test budget;

calculating a testing time limit based on the first system test budget; and

determining the at least one testing constraint as the testing time limit.

15. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by one or more processors to cause the one or more processors to perform operations comprising:

receiving system data associated with a first system, the first system comprising a plurality of system components, wherein the system data comprises component data for each system component in the plurality of system components;

obtaining historical performance data for each system component in the plurality of system components;

determining at least one testing constraint associated with the first system;

determining a test environment for the first system, the test environment comprising a plurality of test cases for the first system based on the system data, the historical performance data, and the at least one testing constraint; and

executing the test suite on the first system, wherein determining the test environment for the first system comprises:

generating, via a machine learning model, a first feature vector comprising a plurality of features extracted from the component data;

plotting the first feature vector in a multi-variate feature space; and

determining the test environment based on a location of the first feature vector in the multi-variate feature space.

16. The computer program product of claim 15 , wherein the operations further comprise:

determining a test time for test environment for the first system based at least in part on the system data, the historical performance data, and the at least one testing constraint.

17. The computer program product of claim 15 , wherein the at least one testing constraint comprises a total testing time limit constraint; and the operations further comprise:

determining an available time period based on the total testing time limit and the testing time for the testing environment; and

collecting learning data on the first system during the available time period.

18. The computer program product of claim 17 , wherein collecting the learning data on the first system comprises:

executing a learning test case on the first system during the available time period; and

adjusting one or more system parameters of the first system during the learning test case execution.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 14, 2021
From: VERBURG, DAVID; ANDORKÓ, GÁBOR; DE LA TORRE DEL RIO, OSCAR ALEJANDRO; FINCH, RICHARD BURTON
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 056847/0876 →
Continuity (1)
Related Publication 20230015477A1 · Jan 19, 2023