Test executive system with automatic expression logging and parameter logging

1. A computer-readable memory medium storing program instructions executable to:

include a plurality of test executive steps in a test executive sequence in response to user input, wherein the plurality of test executive steps includes a first test executive step;

configure each of the test executive steps to call a user-supplied code module in response to user input, wherein the first test executive step is configured to call a first user-supplied code module that takes a first input parameter;

configure one or more expressions for the first test executive step in response to user input, wherein each expression evaluates to a result value to be stored when the first test executive step is executed, wherein the one or more expressions include a first expression that includes the first input parameter for the first user-supplied code module;

execute the test executive sequence, wherein said executing comprises executing each of the test executive steps in the test executive sequence, wherein executing the first test executive step comprises;

invoking execution of the first user-supplied code module called by the first test executive step, wherein invoking execution of the first user-supplied code module includes passing a first value for the first input parameter to the first user-supplied code module;

evaluating the first expression configured for the first test executive step, wherein the first expression evaluates to a second value based on the first value passed for the first input parameter; and

storing the second value in response to evaluating the first expression.

2. The memory medium of claim 1 ,

wherein no user programming is required to implement said evaluating the first expression configured for the first test executive step and said storing the second value.

3. The memory medium of claim 1 ,

wherein storing the second value comprises storing the second value in one or more of a file and/or a database.

4. The memory medium of claim 1 ,

wherein storing the second value comprises storing the second value in a data structure.

5. The memory medium of claim 1 ,

wherein storing the second value comprises logging the second value in an execution report for the test executive sequence.

6. The memory medium of claim 1 ,

wherein evaluating the first expression configured for the first test executive step is performed before said invoking execution of the first user-supplied code module called by the first test executive step.

7. The memory medium of claim 1 ,

wherein evaluating the first expression configured for the first test executive step is performed after said invoking execution of the first user-supplied code module called by the first test executive step.

8. The memory medium of claim 1 ,

wherein the first test executive step has pre-functionality, wherein the pre-functionality comprises functionality to be performed before invoking execution of the first user-supplied code module called by the first test executive step;

wherein executing the first test executive step further comprises executing the pre-functionality of the first test executive step before invoking execution of the first user-supplied code module called by the first test executive step.

9. The memory medium of claim 1 ,

wherein the first test executive step has post-functionality, wherein the post-functionality comprises functionality to be performed after the first user-supplied code module called by the first test executive step finishes executing;

wherein executing the first test executive step further comprises executing the post-functionality of the first test executive step after the first user-supplied code module called by the first test executive step finishes executing.

10. The memory medium of claim 1 ,

wherein said executing the test executive sequence comprises executing the test executive sequence under control of a test executive engine;

wherein the test executive engine is operable to perform said invoking execution of the first user-supplied code module called by the first test executive step, said evaluating the first expression configured for the first test executive step, and said storing the second value.

11. The memory medium of claim 1 , wherein said invoking execution of the first user-supplied code module called by the first test executive step comprises one of:

calling a function in a DLL;

invoking a method of an ActiveX object; or

invoking execution of a graphical program.

12. The memory medium of claim 1 ,

wherein configuring the one or more expressions for the first test executive step requires no user programming.

13. The memory medium of claim 1 , wherein the program instructions are further executable to:

display a graphical user interface specialized for configuring expressions for the first test executive step, wherein the one or more expressions configured for the first test executive step are configured in response to user input to the graphical user interface to specify the one or more expressions.

14. The memory medium of claim 1 ,

wherein the program instructions are further executable to display a graphical user interface for creating the test executive sequence;

wherein said including the plurality of test executive steps in the test executive sequence is performed in response to user input received to the graphical user interface requesting inclusion of the test executive steps in the test executive sequence.

15. The memory medium of claim 1 ,

wherein the first user-supplied code module also takes a first output parameter;

wherein configuring the one or more expressions for the first test executive step further comprises configuring a second expression that includes the first output parameter for the first user-supplied code module;

wherein invoking execution of the first user-supplied code module called by the first test executive step includes passing the first output parameter to the first user-supplied code module, wherein the first user-supplied code module is operable to write a third value into the first output parameter;

wherein the program instructions are further executable to:

evaluate the second expression configured for the first test executive step, wherein the second expression evaluates to a fourth value based on the third value written into the first output parameter; and

store the fourth value in response to evaluating the second expression.

16. The memory medium of claim 15 ,

wherein said passing the first output parameter to the first user-supplied code module comprises passing a first memory location to the first user-supplied code module, wherein said first user-supplied code module writing the third value into the first output parameter comprises the first user-supplied code module writing data into the first memory location;

wherein the second expression evaluates to the fourth value based on the data written into the first memory location by the first user-supplied code module.

17. A method comprising:

including a plurality of test executive steps in a test executive sequence in response to user input, wherein the plurality of test executive steps includes a first test executive step;

configuring each of the test executive steps to call a user-supplied code module in response to user input, wherein the first test executive step is configured to call a first user-supplied code module that takes a first input parameter;

configuring one or more expressions for the first test executive step in response to user input, wherein each expression evaluates to a result value to be stored when the first test executive step is executed, wherein the one or more expressions include a first expression that includes the first input parameter for the first user-supplied code module;

executing the test executive sequence, wherein said executing comprises executing each of the test executive steps in the test executive sequence, wherein executing the first test executive step comprises:

invoking execution of the first user-supplied code module called by the first test executive step, wherein invoking execution of the first user-supplied code module includes passing a first value for the first input parameter to the first user-supplied code module;

evaluating the first expression configured for the first test executive step, wherein the first expression evaluates to a second value based on the first value passed for the first input parameter; and

storing the second value in response to evaluating the first expression.

18. The method of claim 17 ,

wherein storing the step second value comprises storing the second value in one or more of a file and/or a database.

19. The method of claim 17 ,

wherein storing the second value comprises storing the second value in a data structure.

20. The method of claim 17 , wherein the program instructions are further executable to:

display a graphical user interface specialized for configuring expressions for the first test executive step, wherein the one or more expressions configured for the first test executive step are configured in response to user input to the graphical user interface to specify the one or more expressions.

21. The method of claim 17 , wherein the program instructions are further executable to display a graphical user interface for creating the test executive sequence;

wherein said including the plurality of test executive steps in the test executive sequence is performed in response to user input received to the graphical user interface requesting inclusion of the test executive steps in the test executive sequence.

22. The method of claim 17 ,

wherein the first user-supplied code module also takes a first output parameter;

wherein configuring the one or more expressions for the first test executive step further comprises configuring a second expression that includes the first output parameter for the first user-supplied code module;

wherein invoking execution of the first user-supplied code module called by the first test executive step includes passing the first output parameter to the first user-supplied code module, wherein the first user-supplied code module is operable to write a third value into the first output parameter;

wherein the method farther comprises:

evaluating the second expression configured for the first test executive step, wherein the second expression evaluates to a fourth value based on the third value written into the first output parameter; and

storing the fourth value in response to evaluating the second expression.

23. A system comprising:

one or more processors; and

memory storing program instructions;

wherein the program instructions are executable by the one or more processors to:

include a plurality of test executive steps in a test executive sequence in response to user input, wherein the plurality of test executive steps includes a first test executive step;

configure each of the test executive steps to call a user-supplied code module in response to user input, wherein the first test executive step is configured to call a first user-supplied code module that takes a first input parameter;

configure one or more expressions for the first test executive step in response to user input, wherein each expression evaluates to a result value to be stored when the first test executive step is executed, wherein the one or more expressions include a first expression that includes the first input parameter for the first user-supplied code module;

execute the test executive sequence, wherein said executing comprises executing each of the test executive steps in the test executive sequence, wherein executing the first test executive step comprises:

invoking execution of the first user-supplied code module called by the first test executive step, wherein invoking execution of the first user-supplied code module includes passing a first value for the first input parameter to the first user-supplied code module;

evaluating the first expression configured for the first test executive step, wherein the first expression evaluates to a second value based on the first value passed for the first input parameter; and

storing the second value in response to evaluating the first expression.

24. The system of claim 23 ,

wherein the first user-supplied code module also takes a first output parameter;

wherein configuring the one or more expressions for the first test executive step further comprises configuring a second expression that includes the first output parameter for the first user-supplied code module;

wherein invoking execution of the first user-supplied code module called by the first test executive step includes passing the first output parameter to the first user-supplied code module, wherein the first user-supplied code module is operable to write a third value into the first output parameter;

wherein the program instructions are further executable by the one or more processors to:

evaluate the second expression configured for the first test executive step, wherein the second expression evaluates to a fourth value based on the third value written into the first output parameter; and

store the fourth value in response to evaluating the second expression.