Optimized testing system

Inventor: Udo Klein (Eggenstein-Leopoldshafen, DE)

Assignee: SAP SE

G06F11/3684 G06F11/3688 G06F11/3692

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 11,003,574
App. No.: 16/001,687
Granted: May 11, 2021
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Described herein includes a software testing system that optimizes test case scheduling to efficiently and speedily analyze a block of code. The system enhances the performance of software testing by implementing a test controller using test statistics to optimize testing performance. The test controller may use the test statistics to determine relevant test cases to execute, and to provide better and/or faster feedback to users.

Claims (82)

1. A system comprising:

at least one data processor; and

at least one memory storing instructions which, when executed by the at least one data processor, cause operations comprising:

determining, by a test controller, during testing of a first block of code, at least one test statistic has not been collected;

collecting, by the test controller, based on the determination and during the testing of the first block of code, the at least one test statistic, the at least one test statistic comprising an association between a test case from a plurality of test cases and a portion of the first block of code, the association comprising a previous detection, by the test case, of a type of error in the portion of the first block of code, the collecting comprising:

injecting, by the test controller, one or more errors into the first block of code;

testing, by the test controller using a full test suite including each of the plurality of test cases, the first block of code;

detecting, by the test controller, the type of error in the portion of the first block of code, the type of error associated with the injected one or more errors; and

associating, by the test controller, the type of error with the portion of the first block of code;

storing, by the test controller in a database, the at least one test statistic;

generating, by the test controller and based on detection of a commit of a second block of code, a reduced test suite to test the second block of code that includes the portion of the first block of code by at least:

accessing, by the test controller directly from the database, the at least one test statistic;

selecting, by the test controller and based on the accessed at least one test statistic, the test case from the plurality of test cases; and

adding, by the test controller, the test case to the reduced test suite when the association of the accessed at least one test statistic indicates that the test case previously detected the type of error in the portion of the first block of code; and

testing, by the test controller using the reduced test suite including the test case, the second block of code.

2. The system of claim 1 , wherein the operations further comprise:

generating, by the test controller, the at least one test statistic; and

storing the at least one generated test statistic.

3. The system of claim 2 , wherein the generating further comprises: determining, by the test controller, that the second block of code is to be tested upon receipt of a triggering event, the triggering event including an instruction to test a portion of the second block of code, a scheduled test case, and/or the commit of the second block of code.

4. The system of claim 1 , wherein the operations further comprise:

determining, based on the testing the second block of code, that the second block of code includes a detected error in a portion of the second block of code;

generating a ticket associated with the detected error, the ticket including a location of the detected error in the second block of code and user information related to a user from which a client device received the second block of code; and

transmitting the ticket.

5. The system of claim 1 , wherein the at least one test statistic further includes an execution time of the test case of the plurality of test cases, a timestamp indicating a time the test case of the plurality of test cases was run, an identification of a commit of code to a code repository, a detected error detected by the test case of the plurality of test cases, and/or a location of the detected error detected by the test case of the plurality of test cases.

6. The system of claim 1 , wherein the operations further comprise:

determining that a change has occurred in the second block of code; and

retesting the second block of code, the retesting comprising:

detecting that a regression in the second block of code has occurred due at least in part by the change in the second block of code, the regression indicating that at least a portion of the second block of code includes a detected error, the at least the portion of the second block of code being previously error-free,

bisecting, based on the at least one test statistic, the retested second block of code to determine a location of the detected error,

determining that the detected error is located within the at least the portion of the second block of code, and

executing, based on the at least one test statistic, the test case to determine the location of the detected error.

7. The system of claim 6 , wherein the operations further comprise: generating a ticket associated with the detected error in the at least the portion of the second block of code, the ticket including the location of the detected error in the retested second block of code and user information related to a user from which a client device received the change in the second block of code.

8. A computer-implemented method, comprising: