Automatic generation of assert statements for unit test cases
An assert statement generator employs a neural transformer model with attention to generate candidate assert statements for a unit test method that tests a focal method. The neural transformer model is pre-trained with source code programs and natural language text and fine-tuned with test-assert triplets. A test-assert triplet includes a source code snippet that includes: (1) a unit test method with an assert placeholder; (2) the focal method; and (3) a corresponding assert statement. In this manner, the neural transformer model is trained to learn the semantics and statistical properties of a natural language, the syntax of a programming language, and the relationships between the code elements of the programming language and the syntax of an assert statement.
1 . A system, comprising:
a processor and a memory;
wherein the memory includes a program having executable instructions that when executed on the processor performs actions that:
detect a partially-written test method of a source code program under development in a software development environment, wherein the partially-written test method is a partially-written unit test for a focal method without an assert statement;
identify the focal method that is subject of the partially-written test method;
generate a source code snippet including the partially-written test method with an assert placeholder and the focal method, wherein the assert placeholder indicates a position in the partially-written test method for the assert statement;
cause a beam search to generate the assert statement for the partially-written test method, wherein the beam search generates partial sequences of tokens iteratively at each of a plurality of timesteps, wherein the beam search invokes a neural-based model, given the source code snippet, at each of the plurality of timesteps, to generate a probability distribution to identify top-k tokens to be appended to a partial sequence of tokens, wherein the beam search expands search for the asset statement by instantiating new partial sequences with the top-k tokens identified by the probability distribution generated by the neural-based model until a termination condition exists, upon occurrence of the termination condition, a select one of the partial sequences is output as the assert statement for the partially-written test method, wherein the neural-based model is trained to generate the assert statement having been pre-trained on natural language text and source code and fine-tuned with test-assert triplets; and
present the generated assert statement in the software development environment.
2 . The system of claim 1 , wherein the program includes executable instructions that when executed on the processor performs actions that:
identify the focal method automatically from a last method call in the partially-written test method.
3 . The system of claim 1 , wherein the program includes executable instructions that when executed on the processor perform actions that:
identify the focal method automatically from user input.
4 . The system of claim 1 , wherein the termination condition occurs when the probability distribution indicates an end-of-method token as a most likely token to complete the select partial sequence.
5 . The system of claim 1 , wherein the neural-based model is trained to generate candidate assert statements for unit test methods that test a particular focal method.
6 . The system of claim 1 , wherein the program includes executable instructions that when executed on the processor performs actions that:
insert the assert statement into the source code program.
7 . The system of claim 1 , wherein the program includes executable instructions that when executed on the processor performs actions that:
detect the partially-written test method of the source code program from user input forming a beginning of the assert statement.
8 . The system of claim 1 , wherein the program includes executable instructions that when executed on the processor perform actions that:
detect the partially-written test method of a source code program from a user prompt indicating a need for the generated assert statement.
9 . The system of claim 1 , wherein the neural-based model is a neural transformer model with attention.
10 . The system of claim 1 , wherein the neural-based model is provided a context of the focal method, wherein the context comprises a class constructor associated with the focal method, one or more class attributes, or one or more method signatures in a focal class associated with the focal method.
11 . A computer-implemented method, comprising:
obtaining a partially-written test method of a source code program, wherein the partially-written test method is a partially-written unit test for a focal method, wherein the partially-written unit test is without an assert statement;
extracting the focal method that is subject of the partially-written test method;
creating a source code snippet including the partially-written test method with an assert placeholder and the focal method, wherein the assert placeholder indicates a position in the partially-written test method for the assert statement;
causing a beam search to generate the assert statement for the partially-written test method, wherein the beam search generates partial sequences of tokens iteratively at each of a plurality of timesteps, wherein the beam search invokes a neural-based model, given the source code snippet, at each of the plurality of timesteps, to generate a probability distribution to identify top-k tokens to be appended to a partial sequence, wherein the beam search expands search for the assert statement by instantiating new partial sequences with the top-k tokens identified by the probability distribution generated by the neural-based model until a termination condition exists, upon occurrence of the termination condition, a select one of the partial sequences is output as the assert statement for the partially-written test method, wherein the neural-based model is trained to generate the assert statement having been pre-trained on natural language text and source code and fine-tuned with test-assert triplets; and
outputting the generated assert statement in the source code program.
12 . The method of claim 11 , further comprising:
identifying the focal method automatically from a last method call in the partially-written test method.
13 . The method of claim 11 , further comprising:
identifying the focal method automatically from user input.
14 . The method of claim 11 , wherein the termination condition occurs when the probability distribution indicates an end-of-method token as a most likely token to complete the select partial sequence.
15 . The method of claim 11 , further comprising:
displaying select ones of the plurality of candidate assert statements.
16 . The method of claim 15 , further comprising:
inserting the chosen one of the select ones of the plurality of candidate assert statements into the source code program.
17 . The method of claim 11 , further comprising:
detecting the partially-written test method of the source code program from user input forming a beginning of the assert statement.
18 . The method of claim 11 , further comprising:
detecting the partially-written test method of a source code program from a user prompt indicating a need for the generated assert statement.
19 . The method of claim 11 , wherein the neural-based model is a neural transformer model with attention.
20 . The method of claim 11 , wherein the neural-based model is provided a context of the focal method, wherein the context comprises a class constructor associated with the focal method, one or more class attributes, or one or more method signatures in a focal class associated with the focal method.