Training a digital twin in artificial intelligence-defined networking
A system including one or more processors and one or more non-transitory computer-readable media storing computing instructions that, when executed on the one or more processors, perform certain acts. The acts can include generating a digital twin network simulation of a physical computer network controlled through a software-defined-network (SDN) control system. The acts also can include training a routing agent model on the digital twin network simulation using a reinforcement-learning model on traffic that flows through nodes of the digital twin network simulation. The routing agent model includes a machine-learning model. The acts additionally can include deploying the routing agent model, as trained, from the digital twin network simulation to the SDN control system of the physical computer network. Other embodiments are described.
1. A method implemented via execution of computing instructions at one or more processors, the method comprising:
generating a digital twin network simulation of a physical computer network controlled through a software-defined-network (SDN) control system;
training a routing agent model on the digital twin network simulation using a reinforcement-learning model on traffic that flows through nodes of the digital twin network simulation, wherein the routing agent model comprises a machine-learning model; and
deploying the routing agent model, as trained, from the digital twin network simulation to the SDN control system of the physical computer network.
2. The method of claim 1 , wherein the reinforcement-learning model comprises a deep-Q meta-reinforcement learning model.
3. The method of claim 1 , wherein:
the routing agent model comprises a plurality of hierarchical routing agents each controlling a respective hierarchical domain of a plurality of hierarchical domains; and
the reinforcement-learning model comprises a hierarchical reinforcement learning model.
4. The method of claim 3 further comprising:
generating the hierarchical domains using hierarchical algorithmic clustering based on strength and proximity metrics of relationships among the nodes of the digital twin network simulation.
5. The method of claim 1 , wherein the digital twin network simulation is rendered in different portions at different episodic steps of training the routing agent model.
6. The method of claim 1 , wherein connection speeds of the digital twin network simulation are set at a configurable scaled-down ratio of connection speeds of the physical computer network.
7. The method of claim 1 further comprising synthetically generating the traffic based on one or more traffic profiles.
8. The method of claim 7 , wherein at least one of the one or more traffic profiles comprise a fuzzy traffic profile.
9. The method of claim 1 further comprising storing respective versions of the routing agent model, respective versions of network topologies of the physical computer network, and respective versions of traffic patterns captured from the physical computer network as respective configuration items with version control.
10. The method of claim 1 , wherein training the routing agent model of the digital twin network simulation further comprises:
applying a policy-based reward function in the reinforcement-learning model to train the routing agent model to achieve one or more of:
limiting security attacks in the physical computer network;
accommodating changes in the physical computer network;
accommodating failures in the physical computer network;
prioritizing one or more types of traffic routed through the physical computer network;
prioritizing one or more types of applications communicating through the physical computer network;
optimizing device capacity in the physical computer network;
optimizing system capacity in the physical computer network;
optimizing flow of traffic through the physical computer network; or
accounting for variations in demand and consumption in the physical computer network.
11. The method of claim 1 further comprising:
generating a classification of applications from metadata captured from the SDN control system; and
generating, based on the classification, one or more application profiles each being associated with a respective traffic profile.
12. The method of claim 1 , wherein multiple alternative versions of the routing agent model are trained on traffic generated from different traffic profiles.
13. The method of claim 1 , wherein the routing agent model comprises a machine learning model comprising one or more of a neural network model, a random forest model, or a gradient boosted model.
14. A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computing instructions that, when executed on the one or more processors, perform:
generating a digital twin network simulation of a physical computer network controlled through a software-defined-network (SDN) control system;
training a routing agent model on the digital twin network simulation using a reinforcement-learning model on traffic that flows through nodes of the digital twin network simulation, wherein the routing agent model comprises a machine-learning model; and
deploying the routing agent model, as trained, from the digital twin network simulation to the SDN control system of the physical computer network.
15. The system of claim 14 , wherein the reinforcement-learning model comprises a deep-Q meta-reinforcement learning model.
16. The system of claim 14 , wherein:
the routing agent model comprises a plurality of hierarchical routing agents each controlling a respective hierarchical domain of a plurality of hierarchical domains; and
the reinforcement-learning model comprises a hierarchical reinforcement learning model.
17. The system of claim 16 , wherein the computing instructions, when executed on the one or more processors, further perform:
generating the hierarchical domains using hierarchical algorithmic clustering based on strength and proximity metrics of relationships among the nodes of the digital twin network simulation.
18. The system of claim 14 , wherein the digital twin network simulation is rendered in different portions at different episodic steps of training the routing agent model.
19. The system of claim 14 , wherein connection speeds of the digital twin network simulation are set at a configurable scaled-down ratio of connection speeds of the physical computer network.
20. The system of claim 14 , wherein the computing instructions, when executed on the one or more processors, further perform:
synthetically generating the traffic based on one or more traffic profiles.
21. The system of claim 20 , wherein at least one of the one or more traffic profiles comprise a fuzzy traffic profile.
22. The system of claim 14 the computing instructions, when executed on the one or more processors, further perform:
storing respective versions of the routing agent model, respective versions of network topologies of the physical computer network, and respective versions of traffic patterns captured from the physical computer network as respective configuration items with version control.
23. The system of claim 14 , wherein training the routing agent model of the digital twin network simulation further comprises:
applying a policy-based reward function in the reinforcement-learning model to train the routing agent model to achieve one or more of:
limiting security attacks in the physical computer network;
accommodating changes in the physical computer network;
accommodating failures in the physical computer network;
prioritizing one or more types of traffic routed through the physical computer network;
prioritizing one or more types of applications communicating through the physical computer network;
optimizing device capacity in the physical computer network;
optimizing system capacity in the physical computer network;
optimizing flow of traffic through the physical computer network; or
accounting for variations in demand and consumption in the physical computer network.
24. The system of claim 14 , wherein the computing instructions, when executed on the one or more processors, further perform:
generating a classification of applications from metadata captured from the SDN control system; and
generating, based on the classification, one or more application profiles each being associated with a respective traffic profile.
25. The system of claim 14 , wherein multiple alternative versions of the routing agent model are trained on traffic generated from different traffic profiles.
26. The system of claim 14 , wherein the routing agent model comprises a machine learning model comprising one or more of a neural network model, a random forest model, or a gradient boosted model.