Autonomous traffic (self-driving) network with traffic classes and passive and active learning
A method and apparatus for dynamically routing traffic from a first network device to a second network device is described. In one implementation, the method includes sampling one or more qualified packets from an ingress stream using neural network classification criteria, appending a flow encapsulation header with the path information, providing the path information to the path computation engine at user approved time intervals, transmitting the packets to destination network device, and collecting real-time network telemetry data. Real-time network telemetry data is stored in a network health database. When a new network transfer is introduced, the system correctly identifies the characteristics of the transfer, in terms of size, throughput, jitter, latency, loss, and completion times. A corrective action is introduced that inputs the current network health information and transfer characteristics by generating corrective control rules to allow the transfer to be optimally routed (e.g., automatically) through the network.
1. An apparatus for autonomous network traffic management, comprising:
(a) a non-transitory medium storing instructions executable by said one or more processors;
(b) wherein said instructions, when executed by at least one of the processors, perform steps comprising:
(i) detecting and extracting real-time network monitoring data as network traffic profiles from a network, and network health data from the network wherein said network health data comprises a historical database of recent network monitoring data that has been analyzed as endpoint behaviors;
(ii) predicting future network statistics based on the network traffic profiles and the network health data;
(iii) creating traffic classes from patterns in the network traffic profiles and the network health data for use as input to an artificial intelligence instance;
(iv) simulating traffic over the network for each of the traffic classes;
(v) optimizing network path routing and prioritizing traffic based on network optimization objectives directed toward meeting user objectives, by utilizing deep reinforcement learning using the predicted future network statistics; and
(vi) implementing the optimized network path routing over the network for particular network traffic profiles.
2. The non-transitory medium of claim 1 , wherein said acquiring network traffic profiles and said network health data instructions when executed by the processor further performs steps comprising:
(a) inputting a sampling time frame;
(b) collecting network information over the sampling time frame;
(c) outputting the network information into the historical database;
(d) wherein the historical database comprises:
(i) a source in the network;
(ii) a destination in the network;
(iii) a transfer size;
(iv) a transfer duration; and
(v) network health data; and
(e) predicting an optimal solution for source-destination pathways for certain traffic classes.
3. The non-transitory medium of claim 2 , wherein each of the certain traffic classes is selected from a group of traffic classes comprising:
(a) long-living flows that contain multiple TB to multiple PB data transfers lasting 1-200 hours;
(b) short-lived flows that contain short amounts of “bursty” data transfers lasting less than 60 seconds;
(c) low-latency traffic flows that require low latency and minimum packet loss;
(d) priority flows that can be prioritized over other flows in the network;
(e) high throughput flows that require bandwidth greater than 100 GB/s;
(f) intent-based flows that have particular conditions associated with them as deadlines, forbidden paths and required network characteristics; and
(g) default flows with no requirements attached.
4. The non-transitory medium of claim 2 , wherein said network health data is selected from one or more of a group of data comprising:
(a) latency;
(b) loss;
(c) bandwidth; and
(d) jitter.
5. The non-transitory medium of claim 3 , wherein said optimizing network path routing when executed by the processor further performs steps comprising:
(a) training a simulator Graph Neural Network (GNN) to provide the optimal solution for source-destination pathways; and
(b) simulating each of the certain traffic classes on the GNN to output the optimal solution for source-destination pathways.
6. The non-transitory medium of claim 5 , wherein said implementing the optimized network path routing over the network for the particular network traffic profiles when executed by the processor further performs steps comprising:
(a) communicating routing actions directly to network elements; and
(b) utilizing the routing actions to transfer a future data transfer from a designated source to a designated destination according to the optimal solution for source-destination pathways for each of the certain traffic classes of the future data transfer.
7. The non-transitory medium of claim 5 , wherein said simulator Graph Neural Network (GNN) is hardware accelerated.
8. The non-transitory medium of claim 7 , wherein said hardware acceleration is accomplished by utilization of a Raspberry Pi or a field programmable gate array (FPGA).
9. An apparatus for autonomous network traffic management, comprising:
(a) one or more network traffic monitors are coupled to network elements within a network to monitor their activity and having a traffic coordinator which communicates back to the apparatus for autonomous network management;
(b) one or more processors configured to receive input from one or more of the network traffic monitors; and
(c) a non-transitory memory storing instructions executable by the processor;
(d) wherein said instructions, when executed by the processor, perform steps comprising:
(i) detecting and extracting network traffic profiles and network health based on at least one of the network traffic monitors;
(ii) identifying traffic on the network, as network traffic, over a period of time into one or more traffic classes from patterns in the network traffic and generating network health data, over the period of time, for use as input to an artificial intelligence instance;
(iii) storing network traffic information into an historical database;
(iv) providing information from the traffic classes, the network health data and the network traffic information to at least one AI/ML (Artificial Intelligence/Machine Learning) library, wherein the AI/ML library comprises a graph neural network (GNN) component;
(v) predicting future network health from the AI/ML library graph neural network (GNN) component to create a Predicted and Optimal network solution;
(vi) simulating traffic over the network for each of the traffic classes, including applying predictions for future network health characteristics;
(vii) Optimizing network path routing and prioritizing the network traffic based on the prediction and the simulation
(viii) implementing the optimized network routing over the network for particular traffic profiles
(ix) outputting the Predicted and Optimal network solution to a Route Behavior Engine (RBE).
10. The non-transitory medium of claim 9 , wherein said identifying network traffic and network health data instructions when executed by the processor further performs steps comprising:
(a) inputting a sampling time frame;
(b) collecting network information over the sampling time frame;
(c) outputting the network information into the historical database;
(d) wherein the historical database comprises:
(i) a source in the network;
(ii) a destination in the network;
(iii) a transfer size;
(iv) a transfer duration; and
(v) network health data; and
(e) predicting an optimal solution for source-destination pathway for a certain traffic classes.
11. The non-transitory medium of claim 10 , wherein said traffic classes are selected from a group of traffic classes comprising:
(a) long-living flows that contain multiple TB to multiple PB data transfers lasting 1-200 hours;
(b) short-lived flows that contain short amounts of “bursty” data transfers lasting less than 60 seconds;
(c) low-latency traffic flows that require low latency and minimum packet loss;
(d) priority flows that can be prioritized over other flows in the network;
(e) high throughput flows that require bandwidth greater than 100 GB per second;
(f) intent-based flows that have particular conditions associated with them such as deadlines, forbidden paths and required network characteristics; and
(g) default flows with no requirements attached.
12. The non-transitory medium of claim 10 , wherein said network health data is selected from one or more of a group of data comprising:
(a) latency;
(b) loss;
(c) bandwidth; and
(d) jitter.
13. The non-transitory medium of claim 10 , wherein said optimizing network path routing when executed by the processor further performs steps comprising:
(a) training the Graph Neural Network (GNN) to provide the optimal solution for source-destination pathway, and
(b) simulating the certain traffic classes on the GNN to output the optimal solution for source-destination pathway.
14. The non-transitory medium of claim 13 , wherein said implementing the optimized network path routing over the network for the particular network traffic profiles when executed by the processor further performs steps comprising:
(a) communicating routing actions directly to the Route Behavior Engine (RBE); and
(b) utilizing the routing actions to transfer a future data transfer from a designated source to a designated destination according to the optimal solution for source-destination pathway for the certain traffic classes of the future data transfer.
15. The non-transitory medium of claim 13 , wherein said simulator Graph Neural Network (GNN) is hardware accelerated.
16. The non-transitory medium of claim 15 , wherein said hardware acceleration is accomplished by utilization of a Raspberry Pi or a field programmable gate array (FPGA).
17. An apparatus for autonomous network traffic management, comprising:
(a) a non-transitory medium storing instructions executable by one or more processors;
(b) wherein said instructions, when executed by at least one of the processors, perform steps comprising:
(i) providing one or more network optimization goals for a network;
(ii) detecting and extracting real-time network monitoring data as network traffic profiles from a network, and network health data from the network wherein said network health data comprises a historical database of recent network monitoring data that has been analyzed as endpoint behaviors;
(iii) creating traffic classes from patterns in the network traffic profiles and the network health data for use as input to an artificial intelligence instance;
(iv) predicting future network statistics on a pretrained hardware accelerated Graph Neural Network (GNN) based on the network traffic profiles and the network health data;
(v) inputting the future network statistics, the one or more network optimization goals, a network topology, and a network characteristic end point behavior into a Graph Optimizer to output optimized path information;
(vi) simulating traffic over the network for each of the traffic classes; and
(vii) generating an optimized network path routing, and prioritizing traffic based on network optimization objectives, over the network using a Path Computation Engine (PCE), the optimized path information, and the network topology.
18. The non-transitory medium of claim 17 , wherein said future network statistics comprise one or more of: latency, jitter, utilization, and loss.
19. The non-transitory medium of claim 17 , wherein the GNN is periodically re-trained in an asynchronous manner based on network traffic characteristics.
20. The non-transitory medium of claim 17 , wherein the GNN is implemented in software or hardware selected from a set of hardware consisting of: a central processing unit (CPU), a graphical processing unit (GPU), and a field programmable gate array (FPGA).