Method for generating test data from expected product output
A system and method for verifying the output of a system under test is disclosed. An example method begins with receiving a production output from a system under test. A verification data set may be generated from the production output. Then, the verification data set may be provided as input to the system under test. A verification output may be received from the system under test resulting from the system under test ingesting the provided verification data set. The production output and the verification output may be compared with each other to determine whether there are any inconsistencies in the outputs.
1. A computer-implemented method for verifying output of a system under test, the method comprising:
receiving a production output from the system under test;
generating a verification data set from the production output;
providing the verification data set as input to the system under test;
receiving a verification output from the system under test resulting from the system under test ingesting the verification data set; and
comparing the production output with the verification output to determine whether there are any inconsistencies, wherein the production output and the verification output include aggregated metrics.
2. The computer-implemented method of claim 1 , wherein the verification data set is in the form of a log file.
3. The computer-implemented method of claim 1 , further comprising providing information to the system under test in addition to the verification data set for the system under test to produce the verification output.
4. The computer-implemented method of claim 3 , wherein the provided information includes an additional file.
5. The computer-implemented method of claim 3 , wherein the provided information includes information sufficient to generate an additional file.
6. The computer-implemented method of claim 1 , wherein the system under test is a system for collecting advertisement impressions, a social network system that calculates usage and viewing statistics, an online store system that tracks purchases, a website application that monitors traffic, a sales software system monitoring salesperson success, or a scientific software system that monitors experiment outcomes.
7. The computer-implemented method of claim 1 , wherein generating a verification data set includes receiving software code or a template that provides information for converting the production output into a verification data set.
8. The computer-implemented method of claim 1 , wherein generating a verification data set includes iterating through multiple aggregated trait assignments and parsing traits and attributes from the trait assignments to create verification data set entries.
9. A system for verifying the output of a system under test, the system comprising:
one or more processing devices and
one or more storage devices storing instructions that, when executed by the one or more processing devices, cause the one or more processing devices to:
receive a production output from the system under test;
generate a verification data set from the production output;
provide the verification data set as input to the system under test;
receive a verification output from the system under test resulting from the system under test ingesting the verification data set; and
compare the production output with the verification output to determine whether there are any inconsistencies,
wherein the production output and the verification output include aggregated metrics.
10. The system of claim 9 , wherein the verification data set is in the form of a log file.
11. The system of claim 9 , further comprising providing information to the system under test in addition to the verification data set from the system under test to produce the verification output.
12. The system of claim 11 , wherein the provided information includes an additional file.
13. The system of claim 11 , wherein the provided information includes information sufficient to generate an additional file.
14. The system of claim 9 , wherein the system under test is a system for collecting advertisement impressions, a social network that calculates usage and viewing statistics, an online store system that tracks purchases, a website application that monitors traffic, a sales software system monitoring salesperson success, or a scientific software system that monitors experiment outcomes.
15. The system of claim 9 , wherein generating a verification data set includes receiving software code or a template that provides information about converting the production output into a verification data set.
16. The system of claim 9 , wherein generating a verification data set includes iterating through multiple aggregated trait assignments and parsing traits and attributes from the trait assignments to create verification data set entries.