Test executive system with processing of results data in chunks
View Patent ↗Test executive system and method of use. The system includes a test executive engine, configured to execute at least one test executive sequence to test at least one unit under test (UUT), a process model that specifies one or more function sequences for pre-test or post-test functionality for the test executive sequences, and a plug-in framework, configured to selectively incorporate one or more process model plug-in instances in the process model. Each process model plug-in instance specifies at least one respective function sequence for pre-test or post-test functionality for the test executive sequences.
1. A computer-implemented method comprising: utilizing a computer to perform:
creating a test executive sequence, wherein the test executive sequence includes a first plurality of steps to be performed by a test executive engine to test a unit under test (UUT); and
configuring the test executive sequence with a process model, wherein the process model includes a second plurality of steps to be performed by the test executive engine before and/or after executing the first plurality of steps, wherein the process model is configured to accept process model plug-ins to define the second plurality of steps, wherein said configuring the test executive sequence with the process model comprises selecting one or more process model plug-ins for the process model, wherein each process model plug-in includes two or more of the second plurality of steps;
wherein the one or more process model plug-ins include a first process model plug-in configured to store and/or process test result data dispatched to the first process model plug-in in chunks, wherein chunk size is controllable by an auto-tuned threshold that is based on time measured for the dispatch and processing of prior results.
2. The method of claim 1 , wherein chunk size is configurable by user input.
3. The method of claim 1 , wherein chunk size is further controllable by one or more specified criteria, comprising:
one or more programmatic or graphical user interface (GUI) settable options.
4. The method of claim 3 , wherein the one or more programmatic or graphical user interface (GUI) settable options comprise one or more of:
maximum number of results per dispatch; or
maximum time between dispatches.
5. The method of claim 3 , wherein the one or more policies that enforce implementation requirements of the first process model plug-in comprise one or more of:
upon entry to a subsequence; or
upon exit from a subsequence.
6. The method of claim 1 , wherein the auto-tuned threshold comprises a time threshold between chunks that automatically ensures that the dispatch and processing time does not exceed the time threshold the user specified for batching results.
7. The method of claim 1 , wherein the auto-tuned threshold comprises a maximum threshold determined by explicit requirements and policy requirements of a union of all active plug-ins that process results as they are generated.
8. The method of claim 1 , further comprising:
testing the UUT, wherein said testing comprises executing each of the process model plug-ins, comprising:
accumulating and dispatching test result data to the first process model plug-in in chunks; and
the first process model plug-in storing and/or processing the test result data in chunks.
9. The method of claim 1 , further comprising:
receiving user input via a graphical user interface selecting each of the one or more process model plug-ins for the process model.
10. A non-transitory computer-accessible memory medium that stores program instructions executable by one or more processors to implement:
creating a test executive sequence, wherein the test executive sequence includes a first plurality of steps to be performed by a test executive engine to test a unit under test (UUT); and
configuring the test executive sequence with a process model, wherein the process model includes a second plurality of steps to be performed by the test executive engine before and/or after executing the first plurality of steps, wherein the process model is configured to accept process model plug-ins to define the second plurality of steps, wherein said configuring the test executive sequence with the process model comprises selecting one or more process model plug-ins for the process model, wherein each process model plug-in includes two or more of the second plurality of steps;
wherein the one or more process model plug-ins include a first process model plug-in configured to store and/or process test result data dispatched to the first process model plug-in in chunks, wherein chunk size is controllable by an auto-tuned threshold that is based on time measured for the dispatch and processing of prior results.
11. The non-transitory computer-accessible memory medium of claim 10 , wherein chunk size is configurable by user input.
12. The non-transitory computer-accessible memory medium of claim 10 , wherein chunk size is controllable by one or more specified criteria, comprising:
one or more programmatic or graphical user interface (GUI) settable options.
13. The non-transitory computer-accessible memory medium of claim 12 , wherein the one or more programmatic or graphical user interface (GUI) settable options comprise one or more of:
maximum number of results per dispatch; or
maximum time between dispatches.
14. The non-transitory computer-accessible memory medium of claim 12 , wherein the one or more policies that enforce implementation requirements of the first process model plug-in comprise one or more of:
upon entry to a subsequence; or
upon exit from a sub sequence.
15. The non-transitory computer-accessible memory medium of claim 10 , wherein the auto-tuned threshold comprises a time threshold between chunks that automatically ensures that the dispatch and processing time does not exceed the time threshold the user specified for batching results.
16. The non-transitory computer-accessible memory medium of claim 10 , wherein the auto-tuned threshold comprises a maximum threshold determined by explicit requirements and policy requirements of a union of all active plug-ins that process results as they are generated.
17. The non-transitory computer-accessible memory medium of claim 12 , wherein the program instructions are further executable to:
test the UUT, including executing each of the process model plug-ins, comprising:
accumulating and dispatching test result data to the first process model plug-in in chunks; and
the first process model plug-in storing and/or processing the test result data in chunks.
18. The non-transitory computer-accessible memory medium of claim 12 , wherein the program instructions are further executable to:
receive user input via a graphical user interface selecting each of the one or more process model plug-ins for the process model.
19. A system comprising:
one or more processors; and
memory storing program instructions, wherein the program instructions are executable by the one or more processors to implement:
creating a test executive sequence, wherein the test executive sequence includes a first plurality of steps to be performed by a test executive engine to test a unit under test (UUT); and
configuring the test executive sequence with a process model, wherein the process model includes a second plurality of steps to be performed by the test executive engine before and/or after executing the first plurality of steps, wherein the process model is configured to accept process model plug-ins to define the second plurality of steps, wherein said configuring the test executive sequence with the process model comprises selecting one or more process model plug-ins for the process model, wherein each process model plug-in includes two or more of the second plurality of steps;
wherein the one or more process model plug-ins include a first process model plug-in configured to store and/or process test result data dispatched to the first process model plug-in in chunks, wherein chunk size is controllable by an auto-tuned threshold that is based on time measured for the dispatch and processing of prior results.