Machine learning-enabled queue management for network devices
The present disclosure describes apparatuses and methods for machine learning-enabled (ML-enabled) queue management for network devices. In some aspects, an ML-enabled queue manager of a network device initializes a queue management setting with a randomized value and the device processes packets through the queue based on the queue management setting. The ML-enabled queue manager measures a performance metric of the queue and provides, to an ML algorithm, an indication of the queue management setting and an indication of the performance metric of the queue. The ML-enabled queue manager then receives, from the machine learning algorithm, an updated queue management setting and configures the queue with the updated queue management setting to process subsequent packets based on the updated queue management setting. By so doing, the ML-enabled queue manager may tune one or more queue management settings of the queue to optimize performance of the network device.
1 . A method for machine learning-enabled queue management for network devices, comprising:
initializing, with a randomized value, a queue management threshold configured to trigger a queue management event or a queue management policy;
applying the queue management threshold to a queue of a network device;
operating the network device to process packets through the queue based on the queue management threshold, the queue management threshold with the randomized value triggering the queue management event or the queue management policy for the queue through which the packets are processed;
measuring a performance metric of the queue associated with the packets processed through the queue based on the queue management threshold;
providing, to a machine learning algorithm, an indication of the queue management threshold and an indication of the performance metric of the queue;
receiving, from the machine learning algorithm, an updated queue management threshold based on the queue management threshold and the performance metric of the queue; and
applying the updated queue management threshold to the queue of the network device to configure the queue to process subsequent packets based on the updated queue management threshold being configured to trigger the queue management event or queue management policy.
2 . The method as recited in claim 1 , further comprising:
determining a score for the queue management threshold based on the performance metric of the queue, and wherein:
providing the indication of the performance metric of the queue to the machine learning algorithm comprises providing the score for the queue management threshold that is determined based on the performance metric of the queue.
3 . The method as recited in claim 2 , wherein the performance metric is a first performance metric and the method further:
quantizing the first performance metric of the queue to provide a first quantized performance metric; measuring a second performance metric of the queue associated with the packets processed through the queue based on the queue management threshold;
quantizing the second performance metric of the queue to provide a second quantized performance metric; and
concatenating the first quantized performance metric of the queue with the second quantized performance metric of the queue to provide the score for the queue management threshold.
4 . The method as recited in claim 3 , further comprising:
measuring a third performance metric of the queue associated with the packets processed through the queue based on the queue management threshold, the third performance metric being a different type of performance metric than the first performance metric and the second performance metric;
quantizing the third performance metric of the queue to provide a third quantized performance metric; and
concatenating the first quantized performance metric, the second quantized performance metric, and the third quantized performance metric to provide the score.
5 . The method as recited in claim 4 , wherein:
the first quantized performance metric comprises, for a duration of time within a learning phase of the machine learning algorithm, a quantized utilization rate for the queue or a port associated with the queue;
the second quantized performance metric comprises, for the duration of time within the learning phase of the machine learning algorithm, a quantized average occupancy of the queue;
the third quantized performance metric comprises, for the duration of time within the learning phase of the machine learning algorithm, a quantized duration of one or more packet flow control events of the queue; and
the score is provided based on concatenating the quantized utilization rate for the queue or a port associated with the queue, the quantized average occupancy of the queue, and the quantized duration of one or more packet flow control events of the queue.
6 . The method as recited in claim 2 , further comprising:
updating, based on the score for the queue management threshold, a score entry in a performance tracking table that is indexed to a value of the queue management threshold; and
obtaining, from the machine learning algorithm, the updated queue management threshold based at least in part on the score entry in the performance tracking table that is updated.
7 . The method as recited in claim 1 , wherein the performance metric of the queue comprises one of:
a utilization rate for a port associated with the queue;
a utilization rate for the queue; or
an average occupancy of the queue.
8 . The method as recited in claim 1 , wherein the queue management threshold comprises one of:
a threshold for the queue to trigger a priority flow control event;
a threshold for the queue to trigger a packet drop policy; or
a threshold for the queue to trigger a packet marking policy.
9 . The method as recited in claims 8 , wherein the queue management threshold comprises one of:
a duration of one or more priority flow control events initiated by the queue based on the queue management threshold;
a number of packets dropped by the queue based on the queue management threshold and in accordance with the packet drop policy; or
a number of packets marked based on the queue management threshold and in accordance with the packet marking policy.
10 . The method as recited in claim 9 , wherein:
the queue is configured with a priority flow control, PFC, protocol by which the queue initiates the one or more priority flow control events;
the packet drop policy comprises a weighted random early detection, WRED, policy and the threshold comprises a minimum threshold for dropping packets, a maximum threshold for dropping packets, or a maximum probability for dropping packets; or
the packet drop policy comprises an explicit congestion notification, ECN, policy and the threshold comprises a minimum threshold for marking packets, a maximum threshold for marking packets, or a maximum probability for marking packets.
11 . A network device comprising:
one or more network ports;
one or more network interfaces coupled to respective ones of the one or more network ports;
a media access control layer configured to communicate packets through the one or more network interfaces;
multiple queues configured to buffer packets communicated through the one or more network interfaces;
a switch controller configured to route the packets between the multiple queues; and
a machine leaning-enabled (ML-enabled) queue manager associated with the multiple queues and a machine learning, ML, algorithm and configured to:
initialize, with a randomized value and for a queue of the multiple queues, a queue management threshold configured to trigger a queue management policy of the queue;
apply the queue management threshold to the queue of the network device;
operate the network device to process packets through the queue based on the queue management threshold, the queue management threshold triggering the queue management policy for the queue through which the packets are processed;
measure a performance metric of the queue associated with the packets processed through the queue based on the queue management threshold;
provide, to a machine learning algorithm, an indication of the queue management threshold and an indication of the performance metric of the queue;
receive, from the machine learning algorithm, an updated queue management threshold based on the queue management threshold and the performance metric of the queue; and
apply the updated queue management threshold to the queue of the network device to configure the queue to process subsequent packets based on the updated queue management threshold being configured to trigger the queue management policy of the queue.
12 . The network device as recited in claim 11 , wherein the ML-enabled queue manager is further configured to:
determine a score for the queue management threshold based on the performance metric of the queue, and wherein:
to provide the indication of the performance metric of the queue to the machine learning algorithm, the ML-enabled queue manager provides the score for the queue management threshold that is determined based on the performance metric of the queue.
13 . The network device as recited in claim 11 , wherein:
the performance metric of the queue comprises one of a utilization rate for a port associated with the queue, a utilization rate for the queue, or an average occupancy of the queue; and
the queue management threshold comprises one of a threshold for the queue to trigger priority flow control policy, a threshold for the queue to trigger a packet drop policy, or a threshold for the queue to trigger a packet marking policy.
14 . The network device as recited in claim 11 , wherein the ML-enabled queue manager is further configured to:
operate the network device to process the subsequent packets through the queue based on the updated queue management threshold;
measure a second performance metric of the queue associated with the subsequent packets processed through the queue based on the updated queue management threshold;
provide, to the machine learning algorithm, an indication of the updated queue management setting threshold and an indication of the second performance metric of the queue; and
select, based on a randomized value, to:
obtain another updated queue management threshold from the machine learning algorithm by which to update the queue management threshold for processing additional packets; or
initialize the queue management threshold with another randomized value for processing additional packets.
15 . The network device as recited in claim 14 , wherein the ML-enabled queue manager is further configured to:
reduce the randomized value over time effective to increase a likelihood of obtaining updated queue management thresholds from the machine learning algorithm.
16 . A system-on-chip (SoC) comprising:
a media access control layer configured to communicate packets through one or more network interfaces;
multiple queues configured to buffer packets communicated through the one or more network interfaces;
a switch controller configured to route the packets between the multiple queues; and
a machine learning-enabled (ML-enabled) queue manager associated with the multiple queues and a machine learning, ML, algorithm and configured to:
initialize, with a randomized value and for a queue of the multiple queues, a queue management threshold of a queue management policy for the queue, the randomized value reduced over time;
apply the queue management threshold to the queue of the SoC;
operate the SoC to process packets through the queue based on the queue management threshold;
measure a performance metric of the queue associated with the packets processed through the queue based on the queue management threshold;
provide, to a machine learning algorithm, an indication of the queue management threshold and an indication of the performance metric of the queue;
receive, from the machine learning algorithm, an updated queue management threshold based on the queue management threshold and the performance metric of the queue; and
apply the updated queue management threshold to the queue of the SoC to configure the queue to process subsequent packets based on the updated queue management threshold of the queue management policy of the queue.
17 . The SoC as recited in claim 16 , wherein the ML-enabled queue manager is further configured to:
determine a score for the queue management threshold based on the performance metric of the queue, and wherein:
to provide the indication of the performance metric of the queue to the machine learning algorithm, the ML-enabled queue manager provides the score for the queue management threshold that is determined based on the performance metric of the queue.
18 . The SoC as recited in claim 17 , wherein to determine the score for the queue management threshold, the ML-enabled queue manager is configured to:
quantize the performance metric of the queue; or
concatenate the performance metric of the queue with at least one other score for the queue management setting threshold or at least one other performance metric of the queue.
19 . The SoC as recited in claim 16 , wherein:
the performance metric of the queue comprises one of a utilization rate for a port associated with the queue, a utilization rate for the queue, or an average occupancy of the queue; and
the queue management threshold comprises one of a threshold for the queue to trigger a priority flow control event, a threshold for the queue to trigger a packet drop policy, or a threshold for the queue to trigger a packet marking policy.
20 . The SoC as recited in claim 16 , further comprising a processor, a machine learning engine, or an artificial intelligence engine configured to implement the machine learning algorithm.