Determining non-functional requirements (NFR) from a run time environment and incorporating into a development cycle
A computer-implemented method, according to one embodiment, includes analyzing application details associated with a plurality of different timestamp intervals, in response to a determination, from results of analyzing runtime information associated with an application failure event and/or sub-par runtime performance that occurs during a first operational run cycle of an application, that the failure event and/or sub-par runtime performance is caused by a Non-Functional Requirement (NFR) issue. In response to a determination that a first of the timestamp intervals falls within a timestamp associated with the application failure event and/or sub-par runtime performance, first pattern information about the application details associated with the first timestamp interval is collected. The method further includes applying the first pattern information to a weightage algorithm. In response to a determination that test coverage gaps exist in a predetermined collection of test cases, test cases are created based on the first pattern information.
1. A computer-implemented method, comprising:
in response to a determination, from results of analyzing runtime information associated with an application failure event and/or sub-par runtime performance that occurs during a first operational run cycle of an application, that the failure event and/or sub-par runtime performance is caused by a Non-Functional Requirement (NFR) issue, analyzing application details associated with a plurality of different timestamp intervals;
in response to a determination that a first of the timestamp intervals falls within a predetermined timestamp associated with the application failure event and/or sub-par runtime performance, collecting first pattern information about the application details associated with the first timestamp interval;
applying the first pattern information to a predetermined weightage algorithm; and
in response to a determination, based on an output of the predetermined weightage algorithm, that test coverage gaps exist in a predetermined collection of test cases, causing test cases to be created based on the first pattern information for mitigating the NFR issue during a subsequent operational run cycle of the application.
2. The computer-implemented method of claim 1 , wherein the first pattern information is collected using techniques selected from the group consisting of: static code analysis, analysis of an underlying infrastructure configuration provided to a DevOps environment that the first operational run cycle is run in, and analysis of dynamic run time data from the DevOps environment.
3. The computer-implemented method of claim 1 , wherein the first pattern information includes: application details, infrastructure configuration details and operational environment details.
4. The computer-implemented method of claim 3 , wherein the application details are selected from the group consisting of: a number of database tables, a size of columns, a type of the columns, and a type of language; wherein the infrastructure configuration details are selected from the group consisting of: whether a current virtual machine has dedicated central processing unit (CPU) allocated, whether the current virtual machine has shared CPU allocated, an amount of memory allocation, an amount of CPU allocation, and a number of processors dedicated to the current virtual machine; and wherein the operational environment details are selected from the group consisting of: a number of processors that are currently running in a predetermined pattern, a number of operations that are currently running in a predetermined pattern, a number of applications that are currently running in a predetermined pattern, and a maintenance level of the operational environment.
5. The computer-implemented method of claim 1 , comprising: causing the predetermined collection of test cases and the test cases created based on the first pattern information to be used for training a predetermined learning model; and using the trained predetermined learning model for performing a second operational run cycle of the application.
6. The computer-implemented method of claim 1 , wherein applying the application details associated with a first timestamp interval to the predetermined weightage algorithm includes calculating a first coverage weight, wherein the first coverage weight is based on the predetermined collection of test cases and the application details.
7. The computer-implemented method of claim 6 , comprising: comparing the first coverage weight to a second coverage weight, wherein the second coverage weight is based on the predetermined collection of test cases and is not based on the application details; and determining that test coverage gaps exist in the predetermined collection of test cases in response to a determination that results of the comparison reveal that the first coverage weight is less than the second coverage weight.
8. The computer-implemented method of claim 6 , wherein the predetermined weightage algorithm incorporates a plurality of different predetermined factors selected from the group consisting of: application runtime performance requirements, application runtime reliability requirements, application scalability requirements, application availability requirements, and application runtime security requirements.
9. The computer-implemented method of claim 1 , wherein an automated script is used for causing the test cases to be created.
10. A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions readable and/or executable by a computer to cause the computer to:
in response to a determination, from results of analyzing runtime information associated with an application failure event and/or sub-par runtime performance that occurs during a first operational run cycle of an application, that the failure event and/or sub-par runtime performance is caused by a Non-Functional Requirement (NFR) issue, analyze application details associated with a plurality of different timestamp intervals;
in response to a determination that a first of the timestamp intervals falls within a predetermined timestamp associated with the application failure event and/or sub-par runtime performance, collect first pattern information about the application details associated with the first timestamp interval;
apply the first pattern information to a predetermined weightage algorithm; and
in response to a determination, based on an output of the predetermined weightage algorithm, that test coverage gaps exist in a predetermined collection of test cases, cause test cases to be created based on the first pattern information for mitigating the NFR issue during a subsequent operational run cycle of the application.
11. The computer program product of claim 10 , wherein the first pattern information is collected using techniques selected from the group consisting of: static code analysis, analysis of an underlying infrastructure configuration provided to a DevOps environment that the first operational run cycle is run in, and analysis of dynamic run time data from the DevOps environment.
12. The computer program product of claim 10 , wherein the first pattern information includes: application details, infrastructure configuration details and operational environment details.
13. The computer program product of claim 12 , wherein the application details are selected from the group consisting of: a number of database tables, a size of columns, a type of the columns, and a type of language; wherein the infrastructure configuration details are selected from the group consisting of: whether a current virtual machine has dedicated central processing unit (CPU) allocated, whether the current virtual machine has shared CPU allocated, an amount of memory allocation, an amount of CPU allocation, and a number of processors dedicated to the current virtual machine; and wherein the operational environment details are selected from the group consisting of: a number of processors that are currently running in a predetermined pattern, a number of operations that are currently running in a predetermined pattern, a number of applications that are currently running in a predetermined pattern, and a maintenance level of the operational environment.
14. The computer program product of claim 10 , the program instructions readable and/or executable by the computer to cause the computer to: cause the predetermined collection of test cases and the test cases created based on the first pattern information to be used for training a predetermined learning model; and use the trained predetermined learning model for performing a second operational run cycle of the application.
15. The computer program product of claim 10 , wherein applying the application details associated with a first timestamp interval to the predetermined weightage algorithm includes calculating a first coverage weight, wherein the first coverage weight is based on the predetermined collection of test cases and the application details.
16. The computer program product of claim 15 , the program instructions readable and/or executable by the computer to cause the computer to: compare the first coverage weight to a second coverage weight, wherein the second coverage weight is based on the predetermined collection of test cases and is not based on the application details; and determine that test coverage gaps exist in the predetermined collection of test cases in response to a determination that results of the comparison reveal that the first coverage weight is greater than the second coverage weight.
17. The computer program product of claim 15 , wherein the predetermined weightage algorithm incorporates a plurality of different predetermined factors selected from the group consisting of: application runtime performance requirements, application runtime reliability requirements, application scalability requirements, application availability requirements, and application runtime security requirements.
18. The computer program product of claim 10 , wherein an automated script is used for causing the test cases to be created.
19. A system, comprising:
a processor; and
logic integrated with the processor, executable by the processor, or integrated with and executable by the processor, the logic being configured to:
in response to a determination, from results of analyzing runtime information associated with an application failure event and/or sub-par runtime performance that occurs during a first operational run cycle of an application, that the failure event and/or sub-par runtime performance is caused by a Non-Functional Requirement (NFR) issue, analyze application details associated with a plurality of different timestamp intervals;
in response to a determination that a first of the timestamp intervals falls within a predetermined timestamp associated with the application failure event and/or sub-par runtime performance, collect first pattern information about the application details associated with the first timestamp interval;
apply the first pattern information to a predetermined weightage algorithm; and
in response to a determination, based on an output of the predetermined weightage algorithm, that test coverage gaps exist in a predetermined collection of test cases, cause test cases to be created based on the first pattern information for mitigating the NFR issue during a subsequent operational run cycle of the application.
20. The system of claim 19 , wherein the first pattern information is collected using techniques selected from the group consisting of: static code analysis, analysis of an underlying infrastructure configuration provided to a DevOps environment that the first operational run cycle is run in, and analysis of dynamic run time data from the DevOps environment.