Network-based testing service and method of testing in a network
A network-based testing method and service integrated with a tool that publishes one or more tagged test cases with tags being executable to reproduce a sequence of events for a system under test, SUT, caused by an original test case. The method is performed in a network and is intended for testing software or hardware by first creating an original test case for a system under test, SUT, and performing a sequence of events for the original test case for testing it. The tested case is stored and information of the performed sequence of events is tagged to the tested case. The tagged test case is then sent to a service that publishes tagged test cases. The service publishes the tagged case in a way to be reproduced via the service.
1. A network-based testing service in a network environment ( 3 ) comprising:
a server ( 2 ), executing instruction code recorded on a data storage device in communication with the server ( 2 ); and
a tester computer, which has access to a service provided by the server ( 2 ) in communication with the test computer, and which runs software, recorded on a data storage device of the tester computer, that upon execution by the tester computer performs test case processes, the software configured to cause the tester computer
to fetch one or more tested cases,
to automatically tag a test case causing abnormal behavior in a system under test with information on a performed sequence of events for the test case, and
to transmit the tagged test case to the server, wherein the server is configured to publish the tagged test case with tags being selectable to execute the test case by an other tester computer in communication with the server ( 2 ), the other tester computer automatically reproducing the sequence of events for a different system under test.
2. The testing service of claim 1 , wherein a tag of a tagged test case presents information of the exact setting or instructions of the system under test that were used in the test case.
3. The testing service of claim 2 , wherein a tag of said tagged test case presents additional information to the test case, such as results of the executed test case and other information.
4. The testing service of claim 2 , wherein a tag of said tagged test case further presents information of the kind of anomaly or anomalies in the test case, other input data fed to the system under test in the actual test case, the syntax of the exchanged messages or files, and/or the semantic information about the exchanged messages.
5. The testing service of claim 2 , wherein a tag of said tagged test case further presents identification information of the test case, the identification consisting of an index, a hash code or a message digest calculated over input data of the test case, and/or a test group specifying the location of an anomaly in the test case.
6. The testing service of claim 2 , wherein a tag further specifies the expected response for the test case.
7. The testing service of claim 2 , wherein a tag of said tagged test case further contains a description of a problem found by the test case, a link to external information about the problem, and/or attached files.
8. The testing service of claim 1 , wherein the test case performed for a system under test, is either of a sequence of test messages or a file.
9. The testing service of claim 1 , wherein it is shared by connected users.
10. The testing service of claim 1 , wherein the tagged test cases are categorized on the basis of tagged information, on the basis of the type of system under test, on the basis of a connected project, on the basis of the date of storing or performing the test or in an alphabetical order.
11. The testing service of claim 1 , wherein the tagged test cases are selectively published for connected users on the basis of category.
12. A method of testing in a network ( 3 ), carried out by a tester computer and a server ( 2 ) connected thereto, the tester computer being in communication with a service operated by the server ( 2 ) and having software recorded on a data storage device of the tester computer that, upon execution by the tester computer, performs test case processes, the method comprising at least the following steps:
a) at the tester computer, creating an original test case for an original system under test,
b) at the tester computer, performing a sequence of events for the original test case by communicating with the original system under test, and generating a tested case including information of the performed sequence of events and information associated with abnormal behavior in the original system under test,
c) at the tester computer, storing the tested case and automatically tagging the information of the performed sequence of events of said tested case being associated with the abnormal behavior,
d) at the tester computer, sending the tagged test case to the server ( 2 ) that operates the service, the service configured to publish tagged test cases, and
e) at the server, publishing the tagged case, the tagged test being selected and automatically executed by an other tester computer in communication with the server ( 2 ), and at the other test computer, automatically reproducing the sequence of events of the tagged test for a different system under test.
13. The method of testing claimed in claim 12 , wherein in step e) the tagged test case is stored to be available for one or more users connected to the network ( 3 ).
14. The method of testing claimed in claim 13 , further comprising:
f) selecting, by a user connected to the network ( 3 ), a tagged test case,
g) executing the selected test case by reproducing the sequence of events tagged.
15. The method of testing claimed in claim 12 , further comprising:
automatically publishing the tagged test cases to connected users sharing the information of the service.
16. The method of testing claimed in claim 12 , wherein the test case comprises either of a sequence of test messages or a file.
17. The method of testing claimed in claim 12 , further comprising:
tagging in step c) information of the exact setting or instructions of the software run tested that caused the fault.
18. The method of testing claimed in claim 17 , further comprising:
tagging in step c) additional information to the test case, such as results of the executed events and other information.
19. The method of testing claimed in claim 17 , further comprising:
tagging in step c) additional information to the test case of the kind of anomaly or anomalies in the test case, other input data fed to the system under test in an actual test case, the syntax of the exchanged messages or files, and/or the semantic information about the exchanged messages.
20. The method of testing claimed in claim 17 , further comprising:
tagging in step c) additional information to the test case identifying the test case by means of an index, by means of a hash code or a message digest calculated over input data of a test case and/or by means of a test group specifying the location of an anomaly in the test case.
21. The method of testing claimed in claim 17 , further comprising:
tagging in step c) additional information to the test case specifying responses for a test case.
22. The method of testing claimed in claim 17 , further comprising:
h) detecting the behavior of the system under test for the executed and selected test case,
i) comparing the detected behavior of the system under test to the tagged specified response, and
j) detecting a possible change in the behavior.
23. The method of testing claimed in claim 12 , further comprising:
categorizing the tagged test cases on the basis of the tagged information in accordance with the type of system under test, a connected project, or the date of storing or performing the test or in an alphabetical order.
24. The method of testing claimed in claim 12 , further comprising:
selectively publishing the tagged test cases to relevant users sharing the information of the service on the basis of the categorized tag information.
25. The method of testing claimed in claim 12 , further comprising:
correcting or mitigating a flaw in a tested case by means of a selected tagged case.