Evaluation of library test suites using mutation testing

1. A method for evaluating a test suite for a software library comprising:

loading the software library comprising a first class, wherein the first class is an un-mutated class;

generating a mutated software library by adding a first fault to the first class of the software library to obtain a second class, wherein the second class is a mutated class,

wherein the first class of the software library is used by a testing tool to evaluate a test suite,

wherein the first class of the software library comprises a first security call comprising a check for a first permission,

wherein generating the mutated software library further comprises modifying the first security call to further comprise a check for a second permission to obtain a modified security call, and

wherein the first fault is the modified security call;

loading the mutated software library;

executing, using the mutated class, a test in the test suite on the mutated software library to obtain a test result; and

analyzing the test result.

2. The method of claim 1 , further comprising:

removing the mutated class after executing the test.

3. The method of claim 1 , further comprising:

computing a coverage set comprising a plurality of pairs, each pair relating a mutated class of the software library to a corresponding test of the test suite operating on the mutated class;

executing each pair of a plurality of pairs in the coverage set, wherein executing a pair comprises:

loading the corresponding mutated class,

executing, on the corresponding mutated class, the corresponding test to obtain a test result, and

removing the corresponding mutated class prior to executing a subsequent pair.

4. The method of claim 1 , further comprising:

generating, using the software library, a plurality of test scripts prior to loading the mutated software library.

5. The method of claim 1 , further comprising:

identify a plurality of target classes of the software library;

identify a plurality of tests; and

generating a plurality of mutated software libraries based on the plurality of target classes and the plurality of tests.

6. The method of claim 1 , wherein generating the mutated software library further comprises adding a second fault to the software library, wherein the second fault is a second security call being deleted in the software library.

7. The method of claim 1 , wherein generating the mutated software library further comprises adding a second fault to the software library, wherein the second fault is a second security call comprising a modified parameter with a wider scope than an original parameter in the software library.

8. A system for evaluating a test suite for a software library comprising:

memory comprising the software library comprising a first class, wherein the first class is an un-mutated class; and

a computer processor executing a testing tool for causing the computer processor to:

load the software library,

generate a mutated software library by adding a first fault to the first class of the software library to obtain a second class, wherein the second class is a mutated class,

wherein the first class of the software library is used by a testing tool to evaluate a test suite,

wherein the first class of the software library comprises a first security call comprising a check for a first permission,

wherein generating the mutated software library further comprises modifying the first security call to further comprise a check for a second permission to obtain a modified security call, and

wherein the first fault is the modified security call;

load the mutated software library;

execute, using the mutated class, a test in the test suite on the mutated software library to obtain a test result; and

analyze the test result.

9. The system of claim 8 , wherein the testing tool further causes the computer processor to:

compute a coverage set comprising a plurality of pairs, each pair relating a mutated class of the software library to a corresponding test of the test suite operating on the mutated class;

execute each pair of a plurality of pairs in the coverage set, wherein executing a pair comprises:

loading the corresponding mutated class,

executing, on the corresponding mutated class, the corresponding test to obtain a test result, and

removing the corresponding mutated class prior to executing a subsequent pair.

10. The system of claim 8 , wherein generating the mutated software library further comprises adding a second fault to the software library, wherein the second fault is a second security call being deleted in the software library.

11. The system of claim 8 , wherein generating the mutated software library further comprises adding a second fault to the software library, wherein the second fault is a second security call comprising a modified parameter with a wider scope than an original parameter in the software library.

12. A non-transitory computer readable medium for evaluating a test suite for a software library comprising computer readable program code for:

loading the software library comprising a first class, wherein the first class is an un-mutated class;

generating a mutated software library by adding a first fault to the first class of the software library to obtain a second class, wherein the second class is a mutated class,

wherein the first class of the software library is used by a testing tool to evaluate a test suite,

wherein the first class of the software library comprises a first security call comprising a check for a first permission,

wherein generating the mutated software library further comprises modifying the first security call to further comprise a check for a second permission to obtain a modified security call, and

wherein the first fault is the modified security call;

loading the mutated software library;

executing, using the mutated class, a test in the test suite on the mutated software library to obtain a test result; and

analyzing the test result.

13. The non-transitory computer readable medium of claim 12 , further comprising computer readable program code for:

computing a coverage set comprising a plurality of pairs, each pair relating a mutated class of the software library to a corresponding test of the test suite operating on the mutated class;

executing each pair of a plurality of pairs in the coverage set, wherein executing a pair comprises:

loading the corresponding mutated class,

executing, on the corresponding mutated class, the corresponding test to obtain a test result, and

removing the corresponding mutated class prior to executing a subsequent pair.

14. The non-transitory computer readable medium of claim 12 , wherein generating the mutated software library further comprises adding a second fault to the software library, wherein the second fault is a second security call being deleted in the software library.

15. The non-transitory computer readable medium of claim 12 , wherein generating the mutated software library further comprises adding a second fault to the software library, wherein the second fault is a second security call comprising a modified parameter with a wider scope than an original parameter in the software library.