Test case failure with root cause isolation

1. A computer-implemented method comprising:

executing, by a processor, a first test case on a computer system, wherein the first test case comprises a specification of inputs, execution conditions, and expected results for the computer system;

capturing, by the processor, a first state of the computer system during execution of the first test case, the first state of the computer system comprising applications on the computer system running during the execution of the first test case, central processing unit (CPU) utilization for the computer system during the execution of the first test case, and storage information for the computer system;

determining a first result of the first test case in response to executing the first test case;

performing a second execution of the first test case on the system in response to the first result of the first test case being an unexpected result;

capturing a second state of the system during the second execution of the first test case;

determining a second result of the first test case in response to performing the second execution of the first test case;

analyzing the first state and the second state to determine a state similarity score;

categorizing the first result as a false positive based at least in part on:

the state similarity score being above a first threshold; and

a determination that the second result is a different result than the first result.

2. The computer implemented method of claim 1 , further comprising:

categorizing the first result as a software bug based at least in part on:

the state similarity score being below the first threshold; and

a determination that the second result is a same result as the first result.

3. The computer-implemented method of claim 1 , further comprising:

categorizing the first result as a potential software bug based at least in part on:

the state similarity score being above the first threshold; and

a determination that the second result is a same result as the first result.

4. The computer-implemented method of claim 1 , further comprising:

categorizing the first result as an unlikely bug based at least in part on:

the state similarity score being below the first threshold; and

a determination that the second result is a different result than the first result.

5. The computer-implemented method of claim 1 , further comprising:

monitoring a current state of the computer system;

determining a second similarity score between the current state of the computer system and the first state of the computer system;

performing a third execution of the first test case based on the second similarity score being below the first threshold; and

determining a third result of the third execution of the first test case.

6. The computer-implemented method of claim 5 , further comprising:

categorizing the first result as software bug based on a determination that the third result is a same result as the first result.

7. The computer-implemented method of claim 5 , further comprising:

categorizing the first result as unlikely to be a software bug based on a determination that the third result is a different result than the first result.

8. A system comprising:

a memory having computer readable instructions; and

one or more processors for executing the computer readable instructions, the computer readable instructions controlling the one or more processors to perform operations comprising:

executing a first test case on a computer system, wherein the first test case comprises a specification of inputs, execution conditions, and expected results for the computer system;

capturing a first state of the computer system during execution of the first test case, the first state of the computer system comprising applications on the computer system running during the execution of the first test case, central processing unit (CPU) utilization for the computer system during the execution of the first test case, and storage information for the computer system;

determining a first result of the first test case in response to executing the first test case;

performing a second execution of the first test case on the system in response to the first result of the first test case being an unexpected result;

capturing a second state of the system during the second execution of the second test case;

determining a second result of the first test case in response to performing the second execution of the first test case;

analyzing the first state and the second state to determine a state similarity score;

categorizing the first result as a false positive based at least in part on:

the state similarity score being above a first threshold; and

a determination that the second result is a different result than the first result.

9. The system of claim 8 , further comprising:

categorizing the first result as a software bug based at least in part on:

the state similarity score being below the first threshold; and

a determination that the second result is a same result as the first result.

10. The system of claim 8 , further comprising:

categorizing the first result as a potential software bug based at least in part on:

the state similarity score being above the first threshold; and

a determination that the second result is a same result as the first result.

11. The system of claim 8 , further comprising:

categorizing the first result as an unlikely bug based at least in part on:

the state similarity score being below the first threshold; and

a determination that the second result is a different result than the first result.

12. The system of claim 8 , further comprising:

monitoring a current state of the computer system;

determining a second similarity score between the current state of the computer system and the first state of the computer system;

performing a third execution of the first test case based on the second similarity score being below the first threshold; and

determining a third result of the third execution of the first test case.

13. The system of claim 12 , further comprising:

categorizing the first result as software bug based on a determination that the third result is a same result as the first result.

14. The system of claim 12 , further comprising:

categorizing the first result as unlikely to be a software bug based on a determination that the third result is a different result than the first result.

15. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform operations comprising:

executing a first test case on a computer system, wherein the first test case comprises a specification of inputs, execution conditions, and expected results for the computer system;

capturing a first state of the computer system during execution of the first test case, the first state of the computer system comprising applications on the computer system running during the execution of the first test case, central processing unit (CPU) utilization for the computer system during the execution of the first test case, and storage information for the computer system;

determining a first result of the first test case in response to executing the first test case;

performing a second execution of the first test case on the system in response to the first result of the first test case being an unexpected result;

capturing a second state of the system during the second execution of the first test case;

determining a second result of the first test case;

analyzing the first state and the second state to determine a state similarity score;

categorizing the first result as a false positive based at least in part on:

the state similarity score being above a first threshold; and

a determination that the second result is a different result than the first result.

16. The computer program product of claim 15 , further comprising:

categorizing the first result as a software bug based at least in part on:

the state similarity score being below the first threshold; and

a determination that the second result is a same result as the first result.

17. The computer program product of claim 15 , further comprising:

categorizing the first result as a potential software bug based at least in part on:

the state similarity score being above the first threshold; and

a determination that the second result is a same result as the first result.

18. The computer program product of claim 17 , further comprising:

categorizing the first result as an unlikely bug based at least in part on:

the state similarity score being below the first threshold; and

a determination that the second result is a different result than the first result.

19. The computer program product of claim 15 , further comprising:

monitoring a current state of the computer system;

determining a second similarity score between the current state of the computer system and the first state of the computer system;

performing a third execution of the first test case based on the second similarity score being below the first threshold; and

determining a third result of the third execution of the first test case.

20. The computer program product of claim 19 , further comprising:

categorizing the first result as software bug based on a determination that the third result is a same result as the first result; and

categorizing the first result as an unlikely software bug based on a determination that the third result is a different result than the first result.