Code coverage quality estimator
View Patent ↗A method for estimating a quality of code coverage of a test is described. The method includes training a neural network, using the neural network to generate a risk factor for each code element, and determining a coverage quality based on risk factors of executed code elements and risk factors of unexecuted code elements. The neural network is trained by inputting suggestive data as input and error severity data as output. Suggestive data may be data that correlates to a likelihood that a code element contains an error, and the error severity data is an evaluation of a severity of any error that was present in the code element. A coverage quality can be determined based on the risk factors of the code elements tested during the test and the risk factors of the code elements not tested during the test.
1. A computer-implemented method for estimating a quality of code coverage of a test, the estimating being executed by a processor, the method comprising:
training a neural network by placing the neural network in a learning mode and inputting suggestive data as input and error severity data as output, the output being presented to the neural network when the neural network is in the learning mode, the suggestive data comprising data that correlates to a likelihood that a code element contains an error, and the error severity data is an evaluation of a severity of any error that was present in the code element;
using the neural network to generate a risk factor for each code element, the risk factor being a value representing a likelihood and severity of an error in the code element;
identifying code elements tested during the test;
determining a coverage quality based on the risk factors of the code elements tested during the test and the risk factors of the code elements not tested during the test;
storing the coverage quality to a machine readable medium;
wherein the suggestive data and the error severity data being historical data of at least one computer programming projects.
2. A computer-implemented method for estimating a quality of code coverage of a test, the estimating being executed by a processor, the method comprising:
training a neural network by placing the neural network in a learning mode and inputting suggestive data as input and error severity data as output, the output being presented to the neural network when the neural network is in the learning mode, the suggestive data comprising data that correlates to a likelihood that a code element contains an error, and the error severity data is an evaluation of a severity of any error that was present in the code element;
using the neural network to generate a risk factor for each code element, the risk factor being a value representing a likelihood and severity of an error in the code element;
identifying code elements tested during the test;
determining a coverage quality based on the risk factors of the code elements tested during the test and the risk factors of the code elements not tested during the test;
storing the coverage quality to a machine readable medium;
wherein each code element comprises a statement; and
wherein the suggestive data comprises the statement, an age of the statement, and an identifier that identifies a person who authored the statement.
3. A computer-implemented method for estimating a quality of code coverage of a test, the estimating being executed by a processor, the method comprising:
training a neural network by placing the neural network in a learning mode and inputting suggestive data as input and error severity data as output, the output being presented to the neural network when the neural network is in the learning mode, the suggestive data comprising data that correlates to a likelihood that a code element contains an error, and the error severity data is an evaluation of a severity of any error that was present in the code element;
using the neural network to generate a risk factor for each code element, the risk factor being a value representing a likelihood and severity of an error in the code element;
identifying code elements tested during the test;
determining a coverage quality based on the risk factors of the code elements tested during the test and the risk factors of the code elements not tested during the test;
storing the coverage quality to a machine readable medium;
wherein the determining of the coverage quality comprises:
classifying each of the code elements into a risk category according to the risk factors of the code elements; and
determining a code coverage percentage for each risk category.
4. A computer-implemented method for estimating a quality of code coverage of a test, the estimating being executed by a processor, the method comprising:
training a neural network by placing the neural network in a learning mode and inputting suggestive data as input and error severity data as output, the output being presented to the neural network when the neural network is in the learning mode, the suggestive data comprising data that correlates to a likelihood that a code element contains an error, and the error severity data is an evaluation of a severity of any error that was present in the code element;
using the neural network to generate a risk factor for each code element, the risk factor being a value representing a likelihood and severity of an error in the code element;
identifying code elements tested during the test;
determining a coverage quality based on the risk factors of the code elements tested during the test and the risk factors of the code elements not tested during the test;
storing the coverage quality to a machine readable medium;
wherein the determining of the coverage quality comprises:
classifying each of the code elements into one of a plurality of risk categories according to the risk factors of the code elements; and
applying a weight factor to each code element classified in at least one of the risk categories; and
calculating a weighted coverage percentage.
5. The method of claim 4 , wherein the calculating of the weighted coverage percentage includes solving:
(
WF
)
(
MCE
)
+
LCE
(
WF
)
(
MC
)
+
LC
×
100
=
CQ
%
wherein WF is the weight factor, MCE is a number of the code elements that was executed and belong to a high-risk category; LCE is a number of the code elements that was executed and belong to a low-risk category; MC is a total number of code elements in the high-risk category, LC is a total number of code elements in the low-risk category, and CQ % is the weighted coverage percentage.
6. The method of claim 4 , wherein the determining of the coverage quality comprises solving:
∑
RC
=
1
→
n
(
RC
)
(
EC
RC
)
∑
RC
=
1
→
n
(
RC
)
(
C
RC
)
×
100
=
CQ
%
wherein RF is the risk factor; EC RF is a number of code elements that was executed having the risk factor RF, CRF is a total number of code elements having the risk factor RF, and CQ % is the coverage quality expressed as a percentage value.
7. A machine readable medium for estimating a quality of code coverage of a test, the machine readable medium embodying program instructions causing a computer to perform a method comprising:
training a neural network by placing the neural network in a learning mode and inputting suggestive data as input and error severity data as output, the output being presented to the neural network when the neural network is in the learning mode, the suggestive data comprising data that correlates to a likelihood that a code element contains an error, and the error severity data is an evaluation of a severity of any error that was present in the code element, the suggestive data and the error severity data being historical data of at least one computer programming projects;
using the neural network to generate a risk factor for each code element, the risk factor being a value representing a likelihood and severity of an error in the code element;
identifying code elements tested during the test; and
determining a coverage quality based on the risk factors of the code elements tested during the test and the risk factors of the code elements not tested during the test;
wherein each code element comprises a statement;
wherein the suggestive data comprises the statement, an age of the statement, and an identifier that identifies a person who authored the statement.
8. A machine readable medium for estimating a quality of code coverage of a test, the machine readable medium embodying program instructions causing a computer to perform a method comprising:
training a neural network by placing the neural network in a learning mode and inputting suggestive data as input and error severity data as output, the output being presented to the neural network when the neural network is in the learning mode, the suggestive data comprising data that correlates to a likelihood that a code element contains an error, and the error severity data is an evaluation of a severity of any error that was present in the code element, the suggestive data and the error severity data being historical data of at least one computer programming projects;
using the neural network to generate a risk factor for each code element, the risk factor being a value representing a likelihood and severity of an error in the code element;
identifying code elements tested during the test; and
determining a coverage quality based on the risk factors of the code elements tested during the test and the risk factors of the code elements not tested during the test;
wherein the determining of the coverage quality comprises:
classifying each of the code elements into a risk category according to the risk factors of the code elements; and
determining a code coverage percentage for each risk category.
9. A machine readable medium for estimating a quality of code coverage of a test, the machine readable medium embodying program instructions causing a computer to perform a method comprising:
training a neural network by placing the neural network in a learning mode and inputting suggestive data as input and error severity data as output, the output being presented to the neural network when the neural network is in the learning mode, the suggestive data comprising data that correlates to a likelihood that a code element contains an error, and the error severity data is an evaluation of a severity of any error that was present in the code element, the suggestive data and the error severity data being historical data of at least one computer programming projects;
using the neural network to generate a risk factor for each code element, the risk factor being a value representing a likelihood and severity of an error in the code element;
identifying code elements tested during the test; and
determining a coverage quality based on the risk factors of the code elements tested during the test and the risk factors of the code elements not tested during the test;
wherein the determining of the coverage quality comprises:
classifying each of the code elements into one of a plurality of risk categories according to the risk factors of the code elements; and
applying a weight factor to each code element classified in at least one of the risk categories; and
calculating a weighted coverage percentage.
10. The machine readable medium of claim 9 , wherein the calculating a weighted coverage percentage includes solving:
(
WF
)
(
MCE
)
+
LCE
(
WF
)
(
MC
)
+
LC
×
100
=
CQ
%
wherein WF is the weight factor, MCE is a number of the code elements that was executed and belong to a high-risk category; LCE is a number of the code elements that was executed and belong to a low-risk category; MC is a total number of code elements in the high-risk category, LC is a total number of code elements in the low-risk category, and CQ % is the weighted coverage percentage.
11. A computer-based artificial neural network system for estimating a quality of code coverage of a test, the system comprising:
a back propagation neural network defined on one or more computers and configured to be placed in a learning mode wherein suggestive data is received as input and error severity data is received as output, the suggestive data comprising data that correlates to a likelihood that a code element contains an error, and the error severity data is an evaluation of a severity of any error that was present in the code element, the suggestive data and the error severity data being historical data of at least one computer programming projects, the back propagation neural network further being configured to generate a risk factor for each code element when not in the learning mode, the risk factor being a value representing a likelihood and severity of an error in the code element;
a coverage quality calculator defined on the one or more computers, the coverage quality calculator being configured to determine a coverage quality based on the risk factors of code elements tested during a test and risk factors of code elements not tested during the test, and
memory for saving the coverage quality for the test.
12. The system of claim 11 , wherein each code element comprises a statement.
13. The system of claim 12 , wherein the suggestive data comprises the statement, an age of the statement, and an identifier that identifies a person who authored the statement.
14. The system of claim 11 , wherein the coverage quality calculator is configured to determine the coverage quality by classifying each of the code elements into a risk category according to the risk factors of the code elements and determining a code coverage percentage for each risk category.
15. The system of claim 11 , wherein the coverage quality calculator is configured to determine the coverage quality by:
classifying each of the code elements into one of a plurality of risk categories according to the risk factors of the code elements;
applying a weight factor to each code element classified in at least one of the risk categories; and
calculating a weighted coverage percentage.
16. The system of claim 15 , wherein the calculating of the weighted coverage percentage includes solving:
(
WF
)
(
MCE
)
+
LCE
(
WF
)
(
MC
)
+
LC
×
100
=
CQ
%
wherein WF is the weight factor, MCE is a number of the code elements that was executed and belong to a high-risk category; LCE is a number of the code elements that was executed and belong to a low-risk category; MC is a total number of code elements in the high-risk category, LC is a total number of code elements in the low-risk category, and CQ % is the weighted coverage percentage.