Identifying test gaps using code execution paths
Systems and techniques are described for identifying test gaps. A described technique includes identifying production code paths for an application. Each production code path specifies a respective sequence of code of the application that was executed in a production environment. Test code paths are identified for the application. Each test code path specifies a respective sequence of code of the application that was tested in a test environment. The production code paths are compared to the test code paths to identify a set of first test gaps for the application. Each first test gap specifies a respective production code path that is not included in the test code paths. Test gap data specifying the first test gaps for the application can be provided for presentation to a user.
1. A computer-implemented method comprising:
identifying production code paths for an application, each production code path specifying a respective sequence of code of the application that was executed in a production environment;
identifying test code paths for the application, each test code path specifying a respective sequence of code of the application that was tested in a test environment;
comparing the production code paths to the test code paths to identify a set of first test gaps for the application, each first test gap specifying a respective production code path that is not included in the test code paths;
analyzing source code of the application to identity each potential code execution path of the application, each potential code execution path specifying a respective potential sequence of the application code that can be executed during execution of the application;
comparing the potential code execution paths to the test code paths to identify a set of second test gaps for the application, each second test gap specifying a respective potential code execution path that is not included in the test code paths; and
providing test gap data specifying the first test gaps and the second test gaps for the application for presentation to a user.
2. The method of claim 1 , wherein identifying the production code paths for the application comprises analyzing at least one of: a production log for the application, a code dump for the application, or a thread dump for the application.
3. The method of claim 1 , wherein identifying the test code paths for the application comprises monitoring execution of a test operation on the application.
4. The method of claim 1 , further comprising:
generating a first tree data structure that specifies the production code paths; and
generating a second tree data structure that specifies the test code paths,
wherein comparing the production code paths to the test code paths comprises comparing the first tree data structure to the second tree data structure.
5. The method of claim 1 , further comprising:
generating a first tree data structure that specifies the potential code execution paths; and
generating a second tree data structure that specifies the test code paths
wherein comparing the potential code execution paths to the test code paths comprises comparing the first tree data structure to the second tree data structure.
6. A system comprising:
a data processing apparatus; and
a memory apparatus in data communication with the data processing apparatus and storing instructions executable by the data processing apparatus and that upon such execution cause the data processing apparatus to perform operations comprising:
identifying production code paths for an application, each production code path specifying a sequence of code of the application that was executed in a production environment;
identifying test code paths for the application, each test code path specifying a sequence of code of the application that was tested in a test environment;
identifying a set of first test gaps for the application, each first test gap specifying a production code path that is not included in the test code paths;
analyzing source code of the application to identity each potential code execution path of the application, each potential code execution path specifying a respective potential sequence of the application code that can be executed during execution of the application;
comparing the potential code execution paths to the test code paths to identify a set of second test gaps for the application, each second test gap specifying a respective potential code execution path that is not included in the test code paths; and
providing test gap data specifying the first test gaps and the second test gaps for the application.
7. The system of claim 6 , wherein identifying the production code paths for the application comprises analyzing at least one of: a production log for the application, a code dump for the application, or thread dump for the application.
8. The system of claim 6 , wherein identifying the test code paths for the application comprises monitoring execution of a test operation on the application.
9. The system of claim 6 , wherein the operations further comprise:
generating a first tree data structure that specifies the production code paths; and
generating a second tree data structure that specifies the test code paths, wherein
identifying the set of first test gaps for the application comprises comparing the first tree data structure to the second tree data structure.
10. The system of claim 6 , wherein the operations further comprise:
generating a first tree data structure that specifies the potential code execution paths, and
generating a second tree data structure that specifies the test code paths,
wherein comparing the potential code execution paths to the test code paths comprises comparing the first tree data structure to the second tree data structure.
11. A computer storage medium encoded with a computer program, the computer program comprising instructions that when executed by a data processing apparatus cause the data processing apparatus to perform operations comprising:
identifying production code paths for an application, each production code path specifying a sequence of code of the application that was executed in a production environment;
identifying test code paths for the application, each test code path specifying a sequence of code of the application that was tested in a test environment
comparing the production code paths to the test code paths to identify a set of first test gaps for the application, each first test gap specifying a production code path that is not included in the test code paths:
identifying each potential code execution path of the application, each potential code execution path specifying a potential sequence of the application code that can be executed while the application is being executed;
comparing the potential code execution paths to the test code paths to identify a set of second test gaps for the application, each second test gap specifying a potential code execution path that is not included in the test code paths; and
providing test gap data specifying the first test gaps and the second test gaps for the application.
12. The computer storage medium of claim 11 , wherein identifying the production code paths for the application comprises analyzing at least one of: a production log for the application, a code dump for the application, or a thread dump for the application.
13. The computer storage medium of claim 11 , wherein identifying the test code paths for the application comprises monitoring execution of a test operation on the application.
14. The computer storage medium of claim 11 , wherein the operations further comprise:
generating a first tree data structure that specifies the production code paths; and
generating a second tree data structure that specifies the test code paths,
wherein comparing the production code paths to the test code paths comprises comparing the first tree data structure to the second tree data structure.
15. The computer storage medium of claim 11 , wherein identifying each potential code execution path comprises analyzing source code of the application to identify each potential code execution path of the application.
16. The computer storage medium of claim 11 , wherein the operations further comprise:
generating a first tree data structure that specifies the potential code execution paths; and
generating a second tree data structure that specifies the test code paths,
wherein comparing the potential code execution paths to the test code paths comprises comparing the first tree data structure to the second tree data structure.