Granted Patent B2

US 12,282,417 · App. 18/336,757 · Granted Apr 22, 2025

Inventors: Shaktiraj Chauhan (Normal, IL); Nate Shepherd (Bloomington, IL)

Assignee: State Farm Mutual Automobile Insurance Company

G06F11/3688G06F11/3692G06F11/3696G06N20/00

Abstract

Claims (98)

1. A computer-implemented method, comprising:

distributing, by one or more processors, a group of test cases among different test sets, wherein the group of test cases is configured to test interactions between a software application and a database;

executing, by the one or more processors, the different test sets in parallel;

identifying, by the one or more processors, a set of failed test cases comprising test cases that:

were executed during execution of the different test sets in parallel, and

produced test results indicating that the software application failed to pass the test cases;

determining, by the one or more processors, causes of failures of the test cases in the set of failed test cases;

identifying, by the one or more processors, failure codes that correspond to the causes of the failures of the test cases in the set of failed test cases;

determining, by the one or more processors, and based on the failure codes, an execution order for re-execution of the test cases in the set of failed test cases; and

re-executing, by the one or more processors, the test cases in the set of failed test cases based on the execution order.

2. The computer-implemented method of claim 1 , wherein:

executing the different test sets in parallel is associated with a first likelihood of the causes of the failures, and

re-executing the test cases in the set of failed test cases based on the execution order is associated with a second likelihood, less than the first likelihood, of the causes of the failures reoccurring during re-execution of the test cases in the set of failed test cases.

3. The computer-implemented method of claim 1 , wherein:

the execution order is determined based at least in part by a machine learning model, and

the machine learning model is trained, based on historical data indicating successful re-executions of historical failed test cases associated with the failure codes, to determine an arrangement of the execution order that is predicted to reduce likelihoods of the causes of the failures reoccurring during re-execution of the test cases in the set of failed test cases.

4. The computer-implemented method of claim 1 , wherein:

the set of failed test cases comprises:

one or more first test cases associated with a first failure code of the failure codes, and

one or more second test cases associated with a second failure code of the failure codes, and

the execution order causes the one or more processors to:

re-execute the one or more first test cases substantially immediately following failure of the one or more first test cases, based on the first failure code, and

re-execute the one or more second test cases after a period of time following failure of the one or more second test cases, based on the second failure code.

5. The computer-implemented method of claim 4 , wherein:

the first failure code indicates that database locking errors caused the one or more first test cases to fail, and

the execution order is arranged, based on the first failure code, to cause re-execution of the one or more first test cases substantially immediately following the failure of the one or more first test cases.

6. The computer-implemented method of claim 4 , wherein:

the second failure code indicates that missing data errors caused the one or more second test cases to fail, and

the execution order is arranged, based on the second failure code, to delay re-execution of the one or more second test cases for the period of time, following the failure of the one or more second test cases, to reduce likelihoods of the missing data errors re-occurring during re-execution of the one or more second test cases.

7. The computer-implemented method of claim 1 , wherein re-executing the test cases in the set of failed test cases based on the execution order comprises:

distributing, by the one or more processors, the test cases in the set of failed test cases among different retry test sets; and

executing, by the one or more processors, the different retry test sets in parallel.

8. The computer-implemented method of claim 7 , wherein determining the execution order comprises determining different execution orders associated with the different retry test sets.

9. The computer-implemented method of claim 1 , wherein:

the group of test cases was created based on an expectation of individual test cases, of the group of test cases, being executed in a sequential execution order, and

the failures of one or more of the test cases, in the set of failed test cases, are caused at least in part by executing the different test sets in parallel, rather than executing the individual test cases in the sequential execution order.

10. The computer-implemented method of claim 1 , further comprising:

collecting, by the one or more processors, final test results associated with the group of test cases, wherein the final test results indicate whether individual test cases succeeded during at least one of:

execution of the different test sets in parallel, or

re-execution of the set of failed test cases; and

combining, by the one or more processors, the final test results into an aggregated test result report.

11. A computing device, comprising:

one or more processors; and

memory storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

distributing a group of test cases, configured to test interactions between a software application and a database, among at least a first test set and a second test set;

executing the first test set and the second test set in parallel;

identifying a set of failed test cases comprising test cases that:

were executed during execution of the first test set and the second test set in parallel, and

produced test results indicating that the software application failed to pass the test cases;

determining causes of failures of the test cases in the set of failed test cases;

identifying failure codes that correspond to the causes of the failures of the test cases in the set of failed test cases;

determining, based on the failure codes, an execution order for re-execution of the test cases in the set of failed test cases; and

re-executing the test cases in the set of failed test cases based on the execution order.

12. The computing device of claim 11 , wherein the execution order is determined to reduce likelihoods of the causes of the failures, associated with the failure codes, reoccurring during the re-execution of the test cases in the set of failed test cases.

13. The computing device of claim 11 , wherein:

the set of failed test cases comprises:

one or more first test cases associated with a first failure code of the failure codes, and

one or more second test cases associated with a second failure code of the failure codes, and

the execution order causes the one or more processors to:

re-execute the one or more first test cases substantially immediately following failure of the one or more first test cases, based on the first failure code, and

re-execute the one or more second test cases after a period of time following failure of the one or more second test cases, based on the second failure code.

14. The computing device of claim 11 , wherein re-executing the test cases in the set of failed test cases based on the execution order comprises:

distributing the test cases, in the set of failed test cases, among different retry test sets; and

executing the different retry test sets in parallel.

15. The computing device of claim 11 , wherein:

the group of test cases was created based on an expectation of individual test cases, of the group of test cases, being executed in a sequential execution order, and

the failures of one or more of the test cases, in the set of failed test cases, are caused at least in part by executing the first test set and the second test set in parallel, rather than executing the individual test cases in the sequential execution order.

16. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

distributing a group of test cases, configured to test interactions between a software application and a database, among different test sets;

executing the different test sets in parallel;

identifying a set of failed test cases comprising test cases that:

were executed during execution of the different test sets in parallel, and

produced test results indicating that the software application failed to pass the test cases;

determining causes of failures of the test cases in the set of failed test cases;

identifying failure codes that correspond to the causes of the failures of the test cases in the set of failed test cases;

determining, based on the failure codes, an execution order for re-execution of the test cases in the set of failed test cases; and

re-executing the test cases in the set of failed test cases based on the execution order.

17. The one or more non-transitory computer-readable media of claim 16 , wherein the execution order is determined to reduce likelihoods of the causes of the failures, associated with the failure codes, reoccurring during the re-execution of the test cases in the set of failed test cases.

18. The one or more non-transitory computer-readable media of claim 16 , wherein:

the set of failed test cases comprises:

one or more first test cases associated with a first failure code of the failure codes, and

one or more second test cases associated with a second failure code of the failure codes, and

the execution order causes the one or more processors to:

re-execute the one or more first test cases substantially immediately following failure of the one or more first test cases, based on the first failure code, and

re-execute the one or more second test cases after a period of time following failure of the one or more second test cases, based on the second failure code.

19. The one or more non-transitory computer-readable media of claim 16 , wherein re-executing the test cases in the set of failed test cases based on the execution order comprises:

distributing the test cases in the set of failed test cases among different retry test sets; and

executing the different retry test sets in parallel.

20. A system comprising:

means for distributing a group of test cases, configured to test interactions between a software application and a database, among different test sets;

means for executing the different test sets in parallel;

means for identifying a set of failed test cases comprising test cases that:

were executed during execution of the different test sets in parallel, and

produced test results indicating that the software application failed to pass the test cases;

means for determining causes of failures of the test cases in the set of failed test cases;

means for identifying failure codes that correspond to the causes of the failures of the test cases in the set of failed test cases;

means for determining, based on the failure codes, an execution order for re-execution of the test cases in the set of failed test cases; and

means for re-executing the test cases in the set of failed test cases based on the execution order.

Assignments (1)

Continuity (3)

References Cited (80)

Cited By (1)

US 12,743,355