Method, device and computer program product for executing test cases
View Patent ↗A method and device for executing test cases includes obtaining a set of test cases to be executed, and determining a test platform type and a test script associated with each test case in the set of test cases based on a knowledge base. The set of test cases may be divided into a plurality of test subsets or test suites based on the test platform type, and test cases in each test subset executed using the respective test environment and test script. The test suites may be generated automatically based on the knowledge base, and the respective test environment and test script are used for executing each test suite. Automatic generation and execution of the test suites can improve the operation efficiency for test cases.
1. A method for testing a multi-component system, comprising:
maintaining a knowledge base containing test case records for corresponding test cases, the test case record for each test case including a component identifier for a component tested by the test case, and a test platform identifier for a test platform type required for the test case; and
based on an identification of a set of components to be tested, using the knowledge base to identify a set of test cases and group the set of test cases into platform-specific test subsets, by:
first selecting those test cases whose test case records contain component identifiers for the components to be tested;
then grouping the selected test cases into the test subsets based on test platform types according to the test platform identifiers; and
executing the test subsets in parallel on respective test platforms of the required types.
2. The method according to claim 1 , wherein the test case record for each test case further includes runtime information for the test case, and wherein the grouping of the selected test cases is further based on runtime according to the runtime information.
3. The method according to claim 2 , wherein the grouping of the selected test cases into the test subsets is performed such that differences in runtime among all of the test subsets are within a threshold range.
4. The method according to claim 3 , wherein the grouping produces two or more test subsets for a given test platform type in order to bring the respective runtimes of the test subsets into the threshold range.
5. The method according to claim 1 , wherein the executing the test subsets comprises:
executing, by a plurality of test execution engines, the plurality of test subsets in parallel.
6. The method according to claim 1 , wherein the executing the test subsets comprises:
instantiating respective test environments for the test platform types based on the test platform identifiers; and
executing, in each test environment, a plurality of test scripts associated with the corresponding test subset sequentially.
7. The method according to claim 6 , wherein the executing the plurality of test scripts sequentially comprises:
in response to a plurality of faults including a first fault and a second fault occurring during the execution of the test subset, adjusting a first test case associated with the first fault to be after a second test case associated with the second fault in order; and
re-executing, for the adjusted plurality of faults, test cases associated with the plurality of faults.
8. The method according to claim 1 , further comprising:
after completion of the execution of the test subsets, collecting test results from running logs; and
generating a test report based on the collected test results.
9. The method according to claim 1 , wherein the method is used for automatic regression test, and the knowledge base is built by collecting historically executed test cases.
10. A device for executing a test case, comprising:
a processing unit; and
a memory coupled to the processing unit and storing instructions thereon, the instructions, when executed by the processing unit, perform the following acts:
maintaining a knowledge base containing test case records for corresponding test cases, the test case record for each test case including a component identifier for a component tested by the test case, and a test platform identifier for a test platform type required for the test case; and
based on an identification of a set of components to be tested, using the knowledge base to identify a set of test cases and group the set of test cases into platform-specific test subsets, by:
first selecting those test cases whose test case records contain component identifiers for the components to be tested;
then grouping the selected test cases into the test subsets based on test platform types according to the test platform identifiers; and
executing the test subsets in parallel on respective test platforms of the required types.
11. The device according to claim 10 , wherein the test case record for each test case further includes runtime information for the test case, and wherein the grouping of the selected test cases is further based on runtime according to the runtime information.
12. The device according to claim 11 , wherein the grouping of the selected test cases into the test subsets is performed such that differences in runtime among all of the test subsets are within a threshold range.
13. The device according to claim 12 , wherein the grouping produces two or more test subsets for a given test platform type in order to bring the respective runtimes of the test subsets into the threshold range.
14. The device according to claim 10 , wherein the executing the test subsets comprises:
executing, by a plurality of test execution engines, the plurality of test subsets in parallel.
15. The device according to claim 10 , wherein the executing the test subsets comprises:
instantiating respective test environments for the test platform types based on the test platform identifiers; and
executing, in each test environment, a plurality of test scripts associated with the corresponding test subset sequentially.
16. The device according to claim 15 , wherein the executing the plurality of test scripts sequentially comprises:
in response to a plurality of faults including a first fault and a second fault occurring during the execution of the test subset, adjusting a first test case associated with the first fault to be after a second test case associated with the second fault in order; and
re-executing, for the adjusted plurality of faults, test cases associated with the plurality of faults.
17. The device according to claim 10 , wherein the acts further comprise:
after completion of the execution of the test subsets, collecting test results from running logs; and
generating a test report based on the collected test results.
18. The device according to claim 10 , wherein the acts are used for automatic regression test, and the knowledge base is built by collecting historically executed test cases.
19. A computer program product that is tangibly stored on a non-transient computer readable medium and includes machine-executable instructions for testing a multi-component system, the testing comprising:
maintaining a knowledge base containing test case records for corresponding test cases, the test case record for each test case including a component identifier for a component tested by the test case, and a test platform identifier for a test platform type required for the test case; and
based on an identification of a set of components to be tested, using the knowledge base to identify a set of test cases and group the set of test cases into platform-specific test subsets, by:
first selecting those test cases whose test case records contain component identifiers for the components to be tested;
then grouping the selected test cases into the test subsets based on test platform types according to the test platform identifiers; and
executing the test subsets in parallel on respective test platforms of the required types.
20. The computer program product of claim 19 , wherein the test case record for each test case further includes runtime information for the test case, and wherein the grouping of the selected test cases is further based on runtime according to the runtime information.