System and method for dynamically managing compute and I/O resources in data processing systems

1. A method comprising:

providing a thread pool including a plurality of thread groups, each thread group including one or more threads;

providing a CPU pool including a plurality of CPU groups, each CPU group including a plurality of physical CPUs, and each physical CPU including one or more CPU cores;

receiving a plurality of tasks including I/O tasks and compute tasks;

mapping each of the plurality of tasks to at least one thread group in the thread pool;

and

mapping each of the plurality of thread groups in the thread pool to at least one CPU group in the CPU pool;

receiving performance parameters including a data processing rate (DPR) and an I/O processing rate (IOR) of CPU cores of a data processing system;

monitoring the performance parameters including utilization of the plurality of CPU groups as the I/O tasks and the compute tasks are executed;

comparing performance of the I/O tasks and the compute tasks running on the plurality of CPU cores;

dynamically changing the mappings between the plurality of tasks and the plurality of thread groups and between the plurality of thread groups and the plurality of CPU groups based on the performance parameters to balance the performance between the I/O tasks and compute tasks; and

adjusting the mappings between the plurality of tasks and the plurality of thread groups and between the plurality of thread groups and the plurality of CPU groups when the IOR is greater than the DPR.

2. The method of claim 1 , wherein the performance parameters further include saturation of one or more storage devices, existence of blocking operations, and latency-sensitivity of tasks.

3. The method of claim 1 , wherein one or more I/O tasks and one or more compute tasks are mapped to a same thread group.

4. The method of claim 1 , further comprising:

computing a number of CPU cores allocated for the I/O tasks and the compute tasks;

adjusting a number of the thread groups and affinity of the thread groups to the CPU groups based on the number of CPU cores allocated for the I/O tasks and the compute tasks.

5. The method of claim 4 , further comprising increasing or decreasing a number of threads in a thread group by moving threads from a first thread group to a second thread group.

6. The method of claim 4 , further comprising changing the affinity of a thread group to a first CPU group by changing the mapping between the thread group to a second CPU group.

7. The method of claim 1 , further comprising adjusting 1/0 throttle values to balance the IOR and the DPR.

8. A method comprising:

providing a thread pool including a plurality of thread groups, each thread group including one or more threads;

providing a CPU pool including a plurality of CPU groups, each CPU group including a plurality of physical CPUs, and each physical CPU including one or more CPU cores;

receiving a plurality of tasks including I/O tasks and compute tasks;

mapping each of the plurality of tasks to at least one thread group in the thread pool;

and

mapping each of the plurality of thread groups in the thread pool to at least one CPU group in the CPU pool;

monitoring performance parameters including utilization of the plurality of CPU groups as the I/O tasks and the compute tasks are executed;

comparing performance of the I/O tasks and the compute tasks running on the plurality of CPU cores;

dynamically changing the mappings between the plurality of tasks and the plurality of thread groups and between the plurality of thread groups and the plurality of CPU groups based on the performance parameters to balance the performance between the I/O tasks and compute tasks:

providing an I/O queue for processing the I/O tasks and a data queue for processing the compute tasks;

parsing an I/O task and queueing the parsed 1/0 task in the 1/0 queue;

issuing the I/O task if the data queue is not full; and

queuing the I/O task to the data queue if the data queue has a completed I/O task.

9. The method of claim 8 , further comprising:

applying throttle values; and

rescheduling the I/O task.

10. The method of claim 8 , further comprising:

dequeuing data for a compute task from the data queue;

processing the data at the DPR; and

rescheduling the compute task if a mapping of the compute task is changed.

11. A data processing engine comprising:

a system status monitor configured to monitor performance parameters of a data processing system;

a dynamic mapper configured to map a plurality of tasks including I/O tasks and compute tasks, a plurality of thread groups in a thread pool, and a plurality of CPU groups in a CPU pool, each CPU group including a plurality of physical CPUs, and each physical CPU including one or more CPU cores,

wherein the plurality of thread groups including one or more I/O handlers and one or more data processors,

wherein the dynamic mapper is configured to map each of the plurality of tasks to a thread group of the plurality of thread groups in a thread pool, wherein the dynamic mapper is further configured to map each of the plurality of thread groups in the thread pool to at least one CPU group of the plurality of CPU groups in the CPU pool,

wherein the system status monitor is configured to monitor performance parameters including utilization of the plurality of CPU groups as the I/O tasks and the compute tasks are

executed and compare performance of the I/O tasks and the compute tasks running on the plurality of CPU cores, and

wherein the dynamic mapper is further configured to:

dynamically change the mappings between the plurality of tasks and the plurality of thread groups and between the plurality of thread groups and the plurality of CPU groups based on the performance parameters to balance the performance between the I/O tasks and compute tasks;

receive the performance parameters including a data processing rate (DPR) and an I/O processing rate (IOR) of CPU cores of a data processing system; and

adjust mappings between the plurality of tasks and the plurality of thread groups and between the plurality of thread groups and the plurality of CPU groups when the IOR is greater than the DPR.

12. The data processing engine of claim 11 , wherein the performance parameters further include saturation of one or more storage devices of the data processing system, existence of blocking operations, and latency-sensitivity of tasks.

13. The data processing engine of claim 11 , wherein the dynamic mapper maps one or more I/O tasks and one or more compute tasks to a same thread group.

14. The data processing engine of claim 11 , wherein the dynamic mapper is further configured to:

compute a number of CPU cores allocated for the I/O tasks and the compute tasks; and

adjust a number of the thread groups and affinity of the thread groups to the CPU groups based on the number of CPU cores allocated for the I/O tasks and the compute tasks.

15. The data processing engine of claim 11 , wherein the dynamic mapper is further configured to adjusting I/O throttle values to balance the IOR and the DPR.

16. The data processing engine of claim 11 , wherein each of the I/O handlers is configured to:

provide an I/O queue for processing the I/O tasks and a data queue for processing the compute tasks;

parse an I/O task and queueing the parsed I/O task in the I/O queue;

issue the I/O task if the data queue is not full; and

queue the I/O task to the data queue if the data queue has a completed I/O task.

17. The data processing engine of claim 16 , wherein each of the I/O handlers is further configured to: apply throttle values; and reschedule the I/O task.

18. The data processing engine of claim 11 , wherein each of the data processors is configured to:

dequeue data for a compute task from the data queue;

process the data at the DPR; and

reschedule the compute task if a mapping of the compute task is changed.