Language for development of test harness files
View Patent ↗A test harness development component streamlines the design of regression test harnesses for testing text-based software. An auto-recording tool to automatically record the input and output data in an actual run of the testing session and to automatically generate a test harness. The test harness development component provides an auto-executing tool to automatically run the test harness in subsequent regression tests. The test harness language is used to specify the tests, validation rules of the results, and any other rules for running the test of a test harness. The test harness language is human-readable and emulates the manual test procedures with very little overhead.
1. A method, comprising the steps of:
defining and writing a series of statements comprising a test harness file for the automated regression testing of non-graphical software;
one of the series of statements comprising requesting text string data representing a user-given name for a test in the test harness file;
one of the series of statements comprising entering text string data representing test data of the non-graphical software;
one of the series of statements comprising a non-interactive operating system command comprising a test of the test harness file;
one of the series of statements comprising an interactive operating system command used as a test of the test harness file followed by a prompt pattern used by the interactive operating system command;
one of the series of statements comprising user input within the interactive operating system command;
one of the series of statements comprising a definition of a one-time filter statement that alters the output of only a preceding command and reverts to a default filter for filtering subsequent commands; and
comparing the altered output of the preceding command to a benchmark value.
2. The method of claim 1 wherein the test harness file comprises a plurality of tests, one after the other, delineated by user-given names for the tests.
3. The method of claim 1 wherein the user-given name is responsive to a call for a name in a preceding statement.
4. The method of claim 1 wherein a statement in the series of statements comprises either a non-interactive operating system command or an interactive operating system command, and the statement indicates that the output of an command is to be discarded.
5. The method of claim 1 wherein one of the series of statements comprises either a non-interactive operating system command or an interactive operating system command, and the one of the series of statements indicates that an output of the command is to be compared to determine if the command passes or not.
6. The method of claim 1 wherein one of the series of statements comprises a definition of a filter statement that alters an output of a preceding statement.
7. The method of claim 6 wherein the filter statement comprises a default filter statement used throughout the remainder of the test.
8. The method of claim 6 further comprising the step of comparing output of a preceding command to a benchmark value to subsequent matching to determine if the preceding command passes and wherein the filter statement specifies a portion of the output of the preceding command to be suppressed.
9. The method of claim 8 wherein the filter statement specifies a time stamp generated by the preceding command.
10. The method of claim 8 wherein the filter statement specifies the presence or absence of a key word amongst the output generated by the preceding command.
11. The method of claim 6 further comprising the step of comparing output data of a preceding command to a benchmark value to subsequent matching to determine if the preceding command passes or not and wherein the filter statement is used to filter the output of the preceding statement and the benchmark value.
12. The method of claim 1 wherein one of the series of statements comprises a comment line containing text specified by a user.
13. The method of claim 1 wherein the operating system command comprises a command in the native operating system.
14. A method, comprising the steps of:
defining and writing a series of statements comprising a test harness file for the automated regression testing of non-graphical software;
one of the series of statements comprising requesting text string data representing a user-given name for a test in the test harness file;
one of the series of statements comprising entering text string data representing test data of the non-graphical software;
one of the series of statements comprising a non-interactive operating system command comprising a test of the test harness file;
one of the series of statements comprising an interactive operating system command used as a test of the test harness file followed by a prompt pattern used by the interactive operating system command;
one of the series of statements comprising user input within the interactive operating system command;
one of the series of statements comprising a definition of a one-time filter statement that alters the output of only a preceding command and reverts to a default filter for filtering subsequent commands; and
comparing the altered output of the preceding command to a benchmark value.
15. The method of claim 14 wherein the filter statement indicates a portion of the output of a preceding command which is to be discarded.
16. The method of claim 14 wherein the filter statement comprises a that the default filter statement used throughout the remainder of the test.
17. The method of claim 16 wherein the one of the series of statements comprises a definition of a one-time filter statement that alters the output only of a preceding statement, with the remainder of the test being carried out using the default filter.
18. The method of claim 14 wherein the filter statement specifies the presence or absence of a key word amongst the output generated by the preceding command.
19. The method of claim 14 further comprising the step of comparing output data of a preceding command to a benchmark value to subsequent matching to determine if the preceding command passes or not and wherein the filter statement is used to filter the output of the preceding statement and the benchmark value.
20. A method, comprising the steps of:
defining and writing a series of statements comprising a test harness file for the automated regression testing of non-graphical software;
one of the series of statements comprising requesting text string data representing a user-given name for a test in the test harness file;
one of the series of statements comprising entering text string data representing test data of the non-graphical software;
one of the series of statements comprising a non-interactive operating system command comprising a test of the test harness file;
one of the series of statements comprising an interactive operating system command used as a test of the test harness file followed by a prompt pattern used by the interactive operating system command;
one of the series of statements comprising user input within the interactive operating system command;
one of the series of statements comprising a definition of a one-time filter statement that alters the output of only a preceding command and reverts to a default filter for filtering subsequent commands; and
comparing the altered output of the preceding command to a benchmark value.
21. The method of claim 20 wherein the filter statement specifies the presence or absence of a key word amongst the output generated by the preceding command.
22. The method of claim 20 wherein the filter statement comprises a default filter statement used throughout the remainder of the test.