Processing unit selection mechanism

1. A system, comprising:

a plurality of processing units; and

a device including a control circuit configured to receive a plurality of processing tasks in different packet streams, wherein each processing task of the plurality of processing tasks is received as a network packet in a respective packet stream, and wherein the control circuit is further configured to, for each processing task of the plurality of processing tasks:

determine a category of the processing task based on a header of the corresponding network packet, wherein the header comprises one or more fields from which the device determines the category of the processing task as corresponding to a packet type of the network packet, a network protocol of the network packet, a type of service of the network packet, or a combination thereof;

select, from a set of task distribution techniques and based on the category of the processing task, a task distribution technique for distributing the processing task to the plurality of processing units, wherein the control circuit is configured to select the task distribution technique through:

selecting, in some instances, a round robin (RR) or weighted round robin (WRR) distribution technique based on the category indicating that the processing task can be performed out-of-order relative to other processing tasks in the respective packet stream; and

selecting, in other instances, a receive side scaling (RSS) distribution technique based on the category indicating that the processing task must be performed in order with other processing tasks in the respective packet stream;

select, using the selected task distribution technique and from the plurality of processing units, a target processing unit for performing the processing task; and

forward the processing task to the selected target processing unit.

2. The system of claim 1 , wherein the control circuit is configured to:

select a group of processing units from the plurality of processing units, based on the category of the processing task, load status of the plurality of processing units, or both; and

select, using the selected task distribution technique and from the selected group of processing units, the target processing unit for performing the processing task.

3. The system of claim 1 , wherein the control circuit is configured to:

determine load status of the plurality of processing units,

wherein selecting, from the plurality of processing units, the target processing unit includes selecting the target processing unit based on weights of the plurality of processing units, the load status of the plurality of processing units, batch sizes of the plurality of processing units, or a combination thereof.

4. An integrated circuit for distributing processing tasks, the integrated circuit comprising:

a pre-selector circuit configured to:

receive a plurality of processing tasks in different packet streams, wherein each processing task of the plurality of processing tasks is received as a network packet in a respective packet stream; and

for each processing task of the plurality of processing tasks:

determine a category of the processing task based on a header of the corresponding network packet, wherein the header comprises one or more fields from which the pre-selector circuit determines the category of the processing task as corresponding to a packet type of the network packet, a network protocol of the network packet, a type of service of the network packet, or a combination thereof; and

select, from a set of task distribution techniques and based on the category of the processing task, a task distribution technique for distributing the processing task to a group of processing units, wherein the pre-selector circuit is configured to select the task distribution technique through:

selecting, in some instances, a round robin (RR) or weighted round robin (WRR) distribution technique based on the category indicating that the processing task can be performed out-of-order relative to other processing tasks in the respective packet stream; and

selecting, in other instances, a receive side scaling (RSS) distribution technique based on the category indicating that the processing task must be performed in order with other processing tasks in the respective packet stream; and

a scheduler configured to, for each processing task of the plurality of processing tasks, apply the selected task distribution technique to select, from the group of processing units, a target processing unit for performing the processing task.

5. The integrated circuit of claim 4 , wherein the pre-selector circuit is configured to select the group of processing units from a plurality of processing units, based on the category of the processing task, load status of the plurality of processing units, or both.

6. The integrated circuit of claim 5 , wherein:

the pre-selector circuit includes a processing unit table that stores a plurality of lists of processing units;

the pre-selector circuit is configured to select the group of processing units by selecting a list of processing units in the plurality of lists of processing units; and

the pre-selector circuit is configured to modify the plurality of lists of processing units based on the load status of the plurality of processing units.

7. The integrated circuit of claim 4 , wherein the set of task distribution techniques includes the RR or WRR distribution technique, the RSS distribution technique, and a random distribution technique.

8. The integrated circuit of claim 4 , wherein the scheduler is configured to distribute a batch of processing tasks to a processing unit of the group of processing units.

9. The integrated circuit of claim 8 , wherein the pre-selector circuit or the scheduler is configured to modify a batch size of the batch of processing tasks based on categories of the processing tasks, load status of the group of processing units, or both.

10. The integrated circuit of claim 4 , further comprising a load state circuit configured to collect or receive load information of the group of processing units.

11. The integrated circuit of claim 4 , wherein:

the plurality of processing tasks includes a first processing task received as a Transmission Control Protocol (TCP) packet; and

the scheduler is configured to implement the RSS distribution technique using a hash function that uses information in the TCP packet, a packet number of the TCP packet, or both.

12. The integrated circuit of claim 4 , wherein the scheduler is configured to select, from the group of processing units, the target processing unit based on weights or load status of the group of processing units.

13. The integrated circuit of claim 4 , wherein the group of processing units includes central processing units (CPUs), direct memory access (DMA) engines, network processors, queues, neural network accelerators, or a combination thereof.

14. A computer-implemented method, comprising:

receiving a plurality of processing tasks in different packet streams, wherein each processing task of the plurality of processing tasks is received as a network packet in a respective packet stream; and

for each processing task of the plurality of processing tasks:

determining a category of the processing task based on a header of the corresponding network packet, wherein the header comprises one or more fields from which the category of the processing task is determined as corresponding to a packet type of the network packet, a network protocol of the network packet, a type of service of the network packet, or a combination thereof;

selecting, from a set of task distribution techniques and based on the category of the processing task, a task distribution technique for distributing the processing task to a group of processing units, wherein selecting the task distribution technique comprises:

selecting, in some instances, a round robin (RR) or weighted round robin (WRR) distribution technique based on the category indicating that the processing task can be performed out-of-order relative to other processing tasks in the respective packet stream; and

selecting, in other instances, a receive side scaling (RSS) distribution technique based on the category indicating that the processing task must be performed in order with other processing tasks in the respective packet stream; and

selecting, using the selected task distribution technique and from the group of processing units, a target processing unit for performing the processing task.

15. The computer-implemented method of claim 14 , further comprising selecting the group of processing units from a plurality of processing units, based on the category of the processing task, load status of the plurality of processing units, or both.

16. The computer-implemented method of claim 14 , wherein the set of task distribution techniques includes the RR or WRR distribution technique, the RSS distribution technique, and a random distribution technique.

17. The computer-implemented method of claim 14 , further comprising:

determining load status of the group of processing units,

wherein selecting, from the set of task distribution techniques and based on the category of the processing task, the task distribution technique includes selecting the task distribution technique based on both the category of the processing task and the load status of the group of processing units, and

wherein selecting, from the group of processing units, the target processing unit includes selecting the target processing unit based on weights of the group of processing units, the load status of the group of processing units, batch sizes of the group of processing units, or a combination thereof.