Method and system for automatic processing of failed test scenarios

1. A computer implemented method for automatic processing of failed test scenarios, comprising:

receiving descriptions of test scenarios that failed while executed by an automatic testing component; wherein at least 50% of instructions in the test scenarios are expressed by user interaction activities and not code-related language;

forwarding the test scenarios that failed with abnormal termination to a programmer for code change; wherein abnormal termination does not include normal termination with a value different from an expected value;

forwarding the test scenarios that failed without abnormal termination to a non-programmer user for manual evaluation; wherein the manual evaluation comprises manual re-recording of at least one of the test scenarios that failed without abnormal termination, and wherein the non-programmer user holds a role equivalent to at least one of the following: ERP business analyst, business expert, and end-user; and

responsive to identifying a test scenario, from among the test scenarios, which failed without abnormal termination because of a missing prerequisite, generating a human-readable notice to encourage the non-programmer user to re-record one or more additional steps to reveal the missing prerequisite.

2. The computer implemented method of claim 1 , wherein the abnormal termination is selected from the group of: internal system errors, activation errors, and compilation errors.

3. The computer implemented method of claim 2 , wherein the computer runs Java programming language for Enterprise Resource Planning and the internal system errors are Java exceptions.

4. The computer implemented method of claim 2 , wherein the computer runs Advanced Business Application Programming (ABAP) and the internal system errors are ABAP unexpected errors.

5. The computer implemented method of claim 2 , wherein re-recording of scenarios that failed with abnormal termination has less than 20% chance to fix the abnormal termination failure.

6. The computer implemented method of claim 1 , wherein the automatic testing component executes the test scenarios on a data-oriented large-scale software system, and the test scenarios that failed without abnormal termination comprise test scenarios that do not result in predefined expected values because of insufficient description of the test scenarios, while the data-oriented large-scale software system's code is expected to be error-free; and generating a human-readable notice to request confirmation to add execution result to expected results of the test scenario.

7. The computer implemented method of claim 1 , further comprising generating a human-readable notice to suggest a possible value for the missing prerequisite.

8. The computer implemented method of claim 1 , further comprising directing the non-programmer user to re-record a test scenario in a way that enables a test recording and analysis component to identify a functioning parameter for a certain screen.

9. The computer implemented method of claim 1 , wherein the manual evaluation of the test scenarios that failed without abnormal termination comprise activities selected from the group of: indicating that the test scenarios are in order although resulted in error messages, changing preferences indicating that the received messages are acceptable, adding expected results to approve the test scenarios, and re-recording the test scenarios.

10. A non-transitory computer readable medium for use in a computer to automatic processing of failed test scenarios, the computer including a processor, and the non-transitory computer readable medium comprising:

program code for receiving descriptions of test scenarios that failed while executed by an automatic testing component; wherein at least 50% of instructions in the test scenarios are expressed by user interaction activities and not code-related language;

program code for forwarding the test scenarios that failed with abnormal termination as to a programmer for code change; wherein abnormal termination does not include normal termination with a value different from an expected value;

program code for forwarding the test scenarios that failed without abnormal termination to a non-programmer user for manual evaluation; wherein the manual evaluation comprises manual re-recording of at least one of the test scenarios that failed without abnormal termination, and wherein the non-programmer user holds a role equivalent to at least one of the following: ERP business analyst, business expert, and end-user; and

program code for generating a human-readable notice to encourage the non-programmer user to re-record one or more additional steps to reveal a missing prerequisite after identifying a test scenario, from among the test scenarios, which failed without abnormal termination because of the missing prerequisite.

11. The non-transitory computer readable medium of claim 10 , wherein the abnormal termination is selected from the group of: internal system errors, activation errors, and compilation errors.

12. The non-transitory computer readable medium of claim 11 , wherein the computer runs Java programming language for Enterprise Resource Planning and the internal system errors are Java exceptions.

13. The non-transitory computer readable medium of claim 11 , wherein the computer runs Advanced Business Application Programming (ABAP) and the internal system errors are ABAP unexpected errors.

14. The non-transitory computer readable medium of claim 10 , wherein the automatic testing component executes the test scenarios on a data-oriented large-scale software system, and the test scenarios that failed without abnormal termination comprise test scenarios that do not result in predefined expected values because of insufficient description of the test scenarios, while the data-oriented large-scale software system's code is expected to be error-free; and further comprising program code for generating a human-readable notice to request confirmation to add execution result to expected results of the test scenario.

15. The non-transitory computer readable medium of claim 10 , further comprising program code for generating a human-readable notice to suggest a possible value for the missing prerequisite.

16. The non-transitory computer readable medium of claim 10 , further comprising program code for directing the non-programmer user to re-record a test scenario in a way that enables a test recording and analysis component to identify a functioning parameter for a certain screen.

17. The non-transitory computer readable medium of claim 10 , wherein the manual evaluation of the test scenarios that failed without abnormal termination comprise activities selected from the group of: indicating that the test scenarios are in order although resulted in error messages, changing preferences indicating that the received messages are acceptable, adding expected results to approve the test scenarios, and re-recording the test scenarios.

18. A computer system configured to process failed test scenarios automatically, the computer system comprising:

a storage device containing computer usable code; and

a processing unit operable to execute the computer usable code to:

receive descriptions of test scenarios that failed while executed by an automatic testing component; wherein at least 50% of instructions in the test scenarios are expressed by user interaction activities and not code-related language;

forward the test scenarios that failed with abnormal termination to a programmer for code change; wherein abnormal termination does not include normal termination with a value different from an expected value;

forward the test scenarios that failed without abnormal termination to a non-programmer user for manual evaluation; wherein the manual evaluation comprises manual re-recording of at least one of the test scenarios that failed without abnormal termination, and wherein the non-programmer user holds a role equivalent to at least one of the following: ERP business analyst, business expert, and end-user; and

generate a human-readable notice to encourage the non-programmer user to re-record one or more additional steps to reveal a missing prerequisite after identifying a test scenario, from among the test scenarios, which failed without abnormal termination because of the missing prerequisite.

19. The computer system of claim 18 , wherein the abnormal termination is selected from the group of: internal system errors, activation errors, and compilation errors.

20. The computer system of claim 19 , wherein the computer runs Java programming language for Enterprise Resource Planning and the internal system errors are Java exceptions.

21. The computer system of claim 19 , wherein the computer runs Advanced Business Application Programming (ABAP) and the internal system errors are ABAP unexpected errors.

22. The computer system of claim 18 , wherein the processing unit further executes the computer usable code to generate a human-readable notice to encourage the non-programmer user to re-record one or more additional steps to reveal the missing prerequisite.

23. The computer system of claim 18 , wherein the processing unit further executes the computer usable code to direct the non-programmer user to re-record a scenario in a way that enables a test recording and analysis component to identify a functioning parameter for a certain screen.