Granted Patent B2

US 12,743,355 · App. 18/526,667 · Granted Sep 22, 2026

Inventors: Andrew C. M. Hicks (Highland, NY); Ryan Thomas Rawlins (New Paltz, NY); Michael Terrence Cohoon (Fishkill, NY); John S. Werner (Fishkill, NY)

Assignee: International Business Machines Corporation

G06F11/263

Abstract

Claims (70)

1 . A method for management of test case champions, the method comprising:

generating a plurality of fingerprints for a plurality of test cases, wherein each of the plurality of fingerprints is associated with one of the plurality of test cases and identifies a specific code path covered by its associated test case for a System Under Test (SUT);

identifying, based on the plurality of fingerprints and based on version control information for the SUT, one or more test cases in the plurality of test cases that cover a portion of code of the SUT that has changed;

executing, on the SUT, the one or more test cases to generate test execution results, the identified one or more test cases being executed to identify a fault in the SUT;

designating, based on one or more criteria and the test execution results, at least one test case of the identified one or more test cases as a champion test case;

introducing new code to change a portion of code of the SUT previously covered by the champion test case, wherein introducing the new code includes introducing a previously resolved error;

executing the champion test case to determine whether the champion test case detects the previous resolved error;

validating the champion test case when the champion test case detects the previous resolved error; and

demoting the champion test case to a regular test case when the champion test case does not detect the previous resolved error.

2 . The method of claim 1 , wherein the at least one test case is designated as a champion test case if the at least one test case initially results in failure, and then subsequently results in passing after the portion of code of the SUT covered by the at least one test case is changed.

3 . The method of claim 1 , wherein the plurality of fingerprints are generated based on a plurality of breakpoints within a specific code path covered by a corresponding test case.

4 . The method of claim 1 , further comprising:

identifying, based on the version control information, a portion of code of the SUT that has been updated;

identifying, based on the plurality of fingerprints, a champion test case that covers the identified portion of code that has been updated; and

prioritizing the identified champion test case for execution.

5 . The method of claim 1 , further comprising:

identifying, based on the version control information, a portion of code of the SUT that has been deleted; and

identifying, based on the plurality of fingerprints, the champion test case that previously covered the portion of code that has been deleted.

6 . The method of claim 1 , further comprising:

identifying test cases in the plurality of test cases that resulted in failure;

designating the identified test cases that resulted in failure as potential champion test cases; and

storing, for each of the potential champion test cases, an execution result indicating failure, and system parameters and metrics associated with execution of the potential champion test case.

7 . The method of claim 6 , further comprising:

storing, each time one of the potential champion test cases is subsequently executed, an execution result, and system parameters and metrics associated with executing the potential champion test case;

identifying a potential champion test case with a number of failures that exceeds a first threshold; and

comparing the system parameters and metrics for each execution of the identified potential champion test case to determine whether to designate the identified potential champion test case as a corner case champion.

8 . The method of claim 7 , further comprising:

determining whether the number of failures for the identified potential champion test case exceeds a second threshold higher than the first threshold; and

designating the identified potential champion test case as a champion test case when the number of failures exceeds the second threshold.

9 . The method of claim 1 , wherein the plurality of test cases is generated using Combinatorial Test Design (CTD) testing that models inputs to the SUT as a plurality of attributes, and wherein each attribute includes a set of attribute values.

10 . An apparatus for management of test case champions, the apparatus comprising:

a processing device; and

memory operatively coupled to the processing device, wherein the memory stores computer program instructions that, when executed, cause the processing device to:

generate a plurality of fingerprints for a plurality of test cases, wherein each of the plurality of fingerprints is associated with one of the plurality of test cases and identifies a specific code path covered by its associated test case for a System Under Test (SUT);

identify, based on the plurality of fingerprints and based on version control information for the SUT, one or more test cases in the plurality of test cases that cover a portion of code of the SUT that has changed;

execute, on the SUT, the one or more test cases to generate test execution results, the one or more test cases being executed to identify a fault in the SUT;

designate, based on one or more criteria and test execution results, at least one test case of the identified one or more test cases as a champion test case;

introduce new code to change a portion of code of the SUT previously covered by the champion test case, wherein the computer program instructions that, when executed, cause the processing device to introduce the new code further cause the processing device to introduce a previously resolved error;

execute the champion test case to determine whether the champion test case detects the previous resolved error; and

demote the champion test case to a regular test case when the champion test case does not detect the previous resolved error.

11 . The apparatus of claim 10 , wherein the at least one test case is designated as a champion test case if the at least one test case initially results in failure, and then subsequently results in passing after the portion of code of the SUT covered by the at least one test case is changed.

12 . The apparatus of claim 10 , wherein the plurality of fingerprints are generated based on a plurality of breakpoints within a specific code path covered by a corresponding test case.

13 . The apparatus of claim 10 , wherein the memory further stores computer program instructions that, when executed, cause the processing device to:

identify, based on the version control information, a portion of code of the SUT that has been updated;

identify, based on the plurality of fingerprints, a champion test case that covers the identified portion of code that has been updated; and

prioritize the identified champion test case for execution.

14 . The apparatus of claim 10 , wherein the memory further stores computer program instructions that, when executed, cause the processing device to:

identify, based on the version control information, a portion of code of the SUT that has been deleted; and

identify, based on the plurality of fingerprints, the champion test case that previously covered the portion of code that has been deleted.

15 . The apparatus of claim 10 , wherein the memory further stores computer program instructions that, when executed, cause the processing device to:

identify test cases in the plurality of test cases that resulted in failure;

designate the identified test cases that resulted in failure as potential champion test cases; and

store, for each of the potential champion test cases, an execution result indicating failure, and system parameters and metrics associated with execution of the potential champion test case.

16 . The apparatus of claim 15 , wherein the memory further stores computer program instructions that, when executed, cause the processing device to:

store, each time one of the potential champion test cases is subsequently executed, an execution result, and system parameters and metrics associated with executing the potential champion test case;

identify a potential champion test case with a number of failures that exceeds a first threshold; and

compare the system parameters and metrics for each execution of the identified potential champion test case to determine whether to designate the identified potential champion test case as a corner case champion.

17 . The apparatus of claim 16 , wherein the memory further stores computer program instructions that, when executed, cause the processing device to:

determine whether the number of failures for the identified potential champion test case exceeds a second threshold higher than the first threshold; and

designate the identified potential champion test case as a champion test case if the number of failures exceeds the second threshold.

18 . The apparatus of claim 10 , wherein the plurality of test cases is generated using Combinatorial Test Design (CTD) testing that models inputs to the SUT as a plurality of attributes, and wherein each attribute includes a set of attribute values.

19 . A non-transitory computer readable storage medium, wherein the computer readable storage medium comprises computer program instructions that, when executed:

generate a plurality of fingerprints for a plurality of test cases, wherein each of the plurality of fingerprints is associated with one of the plurality of test cases and identifies a specific code path covered by its associated test case for a System Under Test (SUT);

identify, based on the fingerprints and based on version control information for the SUT, one or more test cases in the plurality of test cases that cover a portion of code of the SUT that has changed;

execute, on the SUT, the one or more test cases to generate test execution results, the one or more test cases being executed to identify a fault in the SUT;

designate, based on one or more criteria and test execution results, at least one test case of the identified one or more test cases as a champion test case;

introduce new code to change a portion of code of the SUT previously covered by the champion test case, wherein the computer program instructions that, when executed, cause a processing device to introduce the new code further cause the processing device to introduce a previously resolved error;

execute the champion test case to determine whether the champion test case detects the previous resolved error; and

demote the champion test case to a regular test case when the champion test case does not detect the previous resolved error.

20 . The non-transitory computer readable medium of claim 19 , wherein the at least one test case is designated as a champion test case if the at least one test case initially results in failure, and then subsequently results in passing after the portion of code of the SUT covered by the at least one test case is changed.

Assignments (1)

Continuity (1)

References Cited (11)