Method for controlling congestion in intermittently-connected and lossy computer networks
A method for controlling congestion in intermittently-connected and lossy computer networks comprising: determining, at a local network node, a payoff score for each of a plurality of active flows of network traffic, wherein each active flow consists of a stream of in-transit packets at the local network node that come from a common source and share a common destination, wherein each active flow's payoff score is based on a pricing model that considers both a sojourn time and a position in a queue of each of an active flow's constituent packets; allocating unused buffer space across all active flows in the local network node based on relative traffic loads with a buffer-space allocation (BSA) agent; and controlling a rate at which packets from all active flows are received at the local network node with a hop-by-hop local-flow-control (LFC) agent according to each flow's payoff score.
1. A method for controlling congestion in intermittently-connected and lossy computer networks comprising:
determining, at a local network node, a payoff score for each of a plurality of active flows of network traffic, wherein each active flow consists of a stream of in-transit packets at the local network node that come from a common source and share a common destination, wherein each active flow's payoff score is based on a pricing model that considers both a sojourn time and a position in a queue of each of an active flow's constituent packets;
allocating unused buffer space across all active flows in the local network node based on relative traffic loads with a buffer-space allocation (BSA) agent; and
controlling a rate at which packets from all active flows are received at the local network node with a hop-by-hop local-flow-control (LFC) agent according to each flow's payoff score.
2. The method of claim 1 , wherein the controlling step comprises:
deciding with the LFC agent according to an LFC policy whether or not to perform one of the following actions for a given active flow to mitigate network congestion: reduce a flow speed, pause the given flow, or restart the given flow if paused; and
wherein the LFC policy is learned via a Proximal Point Optimization (PPO) deep reinforcement learning algorithm using a Markov Decision Process (MDP) as a modeling abstraction of the queue dynamics.
3. The method of claim 2 , wherein the BSA agent follows a BSA policy that allocates greater buffer space to active flows having higher payoff scores according to a Markowitz Portfolio Selection problem that takes into consideration a measure of risk that is quantified through a variability of returns received from each active flow.
4. The method of claim 3 , wherein each flow corresponds to one of a plurality of traffic classes, and wherein the given flow inherits prioritization and quality-of-service requirements from the traffic class to which it corresponds.
5. The method of claim 4 , further comprising forcing upstream nodes to adjust a forwarding rate of the given flow if the LFC agent decides to pause the given flow at the local network node.
6. The method of claim 5 , wherein the LFC agent is a computer software module running on the local network node.
7. The method of claim 6 , wherein the BSA agent is a computer software module running on the local network node.
8. The method of claim 1 , wherein the network traffic is underwater network traffic.
9. The method of claim 3 , further comprising assigning each active flow to an individual active queue in which its constituent packets are stored, wherein each active queue is managed by the BSA agent, the LFC agent and a packet scheduler (PS) according to a first in, first out policy and a tail-drop management scheme.
10. The method of claim 9 , wherein each active queue receives a minimal space allocation.
11. The method of claim 10 , further comprising reallocating, with the BSA agent, unused buffer space to a queue associated with a newly arrived traffic flow.
12. The method of claim 11 , further comprising periodically reevaluating and adjusting, with the BSA agent, buffer-space allocations based on traffic characteristics of each flow.
13. The method of claim 12 , further comprising dropping any new packet arriving at the local network node that is assigned to a queue that is full according to the BSA policy and the tail-drop management scheme.
14. The method of claim 13 , wherein packets are forwarded from active queues based on a schedule and transmission order defined by the PS.
15. The method of claim 14 , further comprising dynamically adjusting the PS to accommodate network traffic dynamics and changes in available transmission bandwidth.
16. The method of claim 15 , wherein the PS is configured to always grant requests from neighboring nodes to perform one or more of the following for a given active flow: reduce a bandwidth allocation, pause, and restart.
17. The method of claim 16 , wherein when a given flow's transmission rate is reduced or paused in response to an LFC request, the PS does not reallocate available bandwidth to any other flow.