Shiftleft topology construction and information augmentation using machine learning
A method, a computer system, and a computer program product for a shiftleft topology construction is provided. Embodiments of the present invention may include collecting datasets. Embodiments of the present invention may include extracting topological entities from the datasets. Embodiments of the present invention may include correlating a plurality of data from the topological entities. Embodiments of the present invention may include mapping the topological entities. Embodiments of the present invention may include marking entry points for a plurality of subgraphs of the topological entities. Embodiments of the present invention may include constructing a topology graph.
1. A method for implementing microservices using a shiftleft topology construction, the method comprises:
determining, based on collected datasets, operational characteristics associated with different microservices in a microservice framework to create predictive results for constructing topologies for the different microservices, wherein the determining further comprises training and using a machine learning model to retrieve the collected datasets and learn the operational characteristics;
extracting topological entities from the collected datasets using the machine learning (ML) model, wherein extracting the topological entities includes using machine learning algorithms associated with the machine learning model to identify keywords and topics that are distributed among the collected datasets and to determine pertinent topics among the identified topics associated with the different microservices;
correlating a plurality of data from the topological entities, wherein the correlating further comprises matching a same topological entity from a plurality of different sources associated with the collected datasets and identifying a correlation of the topological entities between the different microservices and to application program interface (API) routes;
mapping the topological entities, wherein the mapping further comprises mapping the topological entities to source code and identifying relationships from the mapped topological entities;
marking entry points for a plurality of subgraphs of the topological entities;
based on the correlating, the mapping, and the marking of the entry points for the topological entities, constructing an entire topology graph using the machine learning model, wherein nodes represent service endpoints and edges represent dependencies between the service endpoints; and
implementing the microservices using the constructed entire topology graph, wherein the implementing further comprises using the machine learning model to detect changes to the microservices based on updated datasets, updating the constructed entire topology graph based on the updated datasets, and subsequently predicting construction of other entire topology graphs and dependencies between the microservices.
2. The method of claim 1 , further comprising:
updating the datasets using static data; and
validating the updated datasets continuously.
3. The method of claim 2 , wherein validating the updated datasets continuously occurs when a configuration change occurs.
4. The method of claim 1 , further comprising:
updating the datasets using real-time data from subject matter experts (SMEs) as an active learning feedback model; and
validating the updated datasets continuously.
5. The method of claim 1 , wherein extracting the topological entities includes extracting information from authorization entities, login entities, frontend entities, elastic search entities and graph database entities.
6. The method of claim 1 , wherein marking the entry points for the plurality of subgraphs includes marking central processing units (CPUs) cycles of subgraphs for each of the entry points.
7. The method of claim 1 , wherein a learning model is used to obtain meta information from the plurality of subgraphs.
8. A computer system for implementing microservices using a shiftleft topology construction, the computer system comprises:
one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage media, and program instructions stored on at least one of the one or more computer-readable tangible storage media for execution by at least one of the one or more processors via at least one of the one or more computer-readable memories, wherein the computer system is capable of performing a method comprising:
determining, based on collected datasets, operational characteristics associated with different microservices in a microservice framework to create predictive results for constructing topologies for the different microservices, wherein the determining further comprises training and using a machine learning model to retrieve the collected datasets and learn the operational characteristics;
extracting topological entities from the collected datasets using the machine learning (ML) model, wherein extracting the topological entities includes using machine learning algorithms associated with the machine learning model to identify keywords and topics that are distributed among the collected datasets and to determine pertinent topics among the identified topics associated with the different microservices;
correlating a plurality of data from the topological entities, wherein the correlating further comprises matching a same topological entity from a plurality of different sources associated with the collected datasets and identifying a correlation of the topological entities between the different microservices and to application program interface (API) routes;
mapping the topological entities, wherein the mapping further comprises mapping the topological entities to source code and identifying relationships from the mapped topological entities;
marking entry points for a plurality of subgraphs of the topological entities;
based on the correlating, the mapping, and the marking of the entry points for the topological entities, constructing an entire topology graph using the machine learning model, wherein nodes represent service endpoints and edges represent dependencies between the service endpoints; and
implementing the microservices using the constructed entire topology graph, wherein the implementing further comprises using the machine learning model to detect changes to the microservices based on updated datasets, updating the constructed entire topology graph based on the updated datasets, and subsequently predicting construction of other entire topology graphs and dependencies between the microservices.
9. The computer system of claim 8 , further comprising:
updating the datasets using static data; and
validating the updated datasets continuously.
10. The computer system of claim 9 , wherein validating the updated datasets continuously occurs when a configuration change occurs.
11. The computer system of claim 8 , further comprising:
updating the datasets using real-time data from subject matter experts (SMEs) as an active learning feedback model; and
validating the updated datasets continuously.
12. The computer system of claim 8 , wherein extracting the topological entities includes extracting information from authorization entities, login entities, frontend entities, elastic search entities and graph database entities.
13. The computer system of claim 8 , wherein marking the entry points for the plurality of subgraphs includes marking central processing units (CPUs) cycles of subgraphs for each of the entry points.
14. The computer system of claim 8 , wherein a learning model is used to obtain meta information from the plurality of subgraphs.
15. A computer program product for implementing microservices using a shiftleft topology construction, the computer program product comprises:
one or more computer-readable tangible storage media and program instructions stored on at least one of the one or more computer-readable tangible storage media, the program instructions executable by a processor to cause the processor to perform a method comprising:
determining, based on collected datasets, operational characteristics associated with different microservices in a microservice framework to create predictive results for constructing topologies for the different microservices, wherein the determining further comprises training and using a machine learning model to retrieve the collected datasets and learn the operational characteristics;
extracting topological entities from the collected datasets using the machine learning (ML) model, wherein extracting the topological entities includes using machine learning algorithms associated with the machine learning model to identify keywords and topics that are distributed among the collected datasets and to determine pertinent topics among the identified topics associated with the different microservices;
correlating a plurality of data from the topological entities, wherein the correlating further comprises matching a same topological entity from a plurality of different sources associated with the collected datasets and identifying a correlation of the topological entities between the different microservices and to application program interface (API) routes;
mapping the topological entities, wherein the mapping further comprises mapping the topological entities to source code and identifying relationships from the mapped topological entities;
marking entry points for a plurality of subgraphs of the topological entities;
based on the correlating, the mapping, and the marking of the entry points for the topological entities, constructing an entire topology graph using the machine learning model, wherein nodes represent service endpoints and edges represent dependencies between the service endpoints; and
implementing the microservices using the constructed entire topology graph, wherein the implementing further comprises using the machine learning model to detect changes to the microservices based on updated datasets, updating the constructed entire topology graph based on the updated datasets, and subsequently predicting construction of other entire topology graphs and dependencies between the microservices.
16. The computer program product of claim 15 , further comprising:
updating the datasets using static data; and
validating the updated datasets continuously.
17. The computer program product of claim 15 , further comprising:
updating the datasets using real-time data from subject matter experts (SMEs) as an active learning feedback model; and
validating the updated datasets continuously.
18. The computer program product of claim 15 , wherein extracting the topological entities includes extracting information from authorization entities, login entities, frontend entities, elastic search entities and graph database entities.
19. The computer program product of claim 15 , wherein a learning model is used to obtain meta information from the plurality of subgraphs.