Granted Patent B2

US 11,385,931 · App. 16/887,573 · Granted Jul 12, 2022

Inventors: Jinpeng Liu (Shanghai, CN); Jin Li (Shanghai, CN)

Assignee: EMC IP Holding Company LLC

G06F9/4881G06F9/3838G06F9/5083

Abstract

Claims (71)

1. A method for processing a computing job, comprising:

determining a first dependency relationship between a plurality of computing tasks included in a to-be-processed computing job;

determining, based on the first dependency relationship and demands of the plurality of computing tasks for computing resources, a group of computing tasks for combination from the plurality of computing tasks; and

combining the group of computing tasks into a target computing task;

wherein determining the group of computing tasks from the plurality of computing tasks comprises:

determining, for each computing task of the plurality of computing tasks and based on the first dependency relationship, the number of computing tasks associated with at least one of an input or an output of a corresponding computing task;

determining a first group of candidate tasks from the plurality of computing tasks, wherein the number determined for each candidate task is less than a threshold number;

determining, based on demands of the first group of candidate tasks for the computing resources, a second group of candidate tasks for combination from the first group of candidate tasks; and

selecting the group of computing tasks from the second group of candidate tasks.

2. The method according to claim 1 , further comprising:

determining, based on the first dependency relationship, a second dependency relationship between the target computing task and computing tasks that are other than the group of computing tasks in the plurality of computing tasks.

3. The method according to claim 1 , wherein determining the second group of candidate tasks from the first group of candidate tasks comprises:

determining the second group of candidate tasks by removing a candidate task with a demand for the computing resources exceeding a first threshold from the first group of candidate tasks.

4. The method according to claim 1 , wherein selecting the group of computing tasks from the second group of candidate tasks comprises:

determining, based on the first dependency relationship, a first candidate task and a second candidate task in the second group of candidate tasks, wherein the first candidate task does not depend on other candidate tasks in the second group of candidate tasks, and the second candidate task depends on the first candidate task;

estimating a first demand of a first combined task that combines the first candidate task and the second candidate task for the computing resources; and

selecting, if it is determined that the estimated first demand is less than a second threshold, the first candidate task and the second candidate task as at least a part of the group of computing tasks.

5. The method according to claim 4 , further comprising:

estimating, if it is determined that the second group of candidate tasks includes a third candidate task depending on the second candidate task, a second demand of a second combined task that combines the first combined task and the third candidate task for the computing resources; and

adding, if it is determined that the estimated second demand is less than the second threshold, the third candidate task into the group of computing tasks.

6. The method according to claim 4 , further comprising:

determining, if it is determined that a third candidate task depending on the second candidate task is not included in the second group of candidate tasks, the first candidate task and the second candidate task as the group of computing tasks.

7. The method according to claim 1 , wherein determining the first dependency relationship comprises:

obtaining a computation graph representing the computing job, a vertex in the computation graph corresponding to a corresponding computing task among the plurality of computing tasks; and

determining the first dependency relationship from the computation graph.

8. The method according to claim 1 , wherein combining the group of computing tasks into the target computing task comprises:

obtaining a group of generation logics corresponding to the group of computing tasks, each generation logic in the group of generation logics being used for generating an executable instruction for a corresponding computing task; and

combining the group of generation logics into a target logic for generating an executable instruction for the target computing task.

9. The method according to claim 1 , wherein the computing resources comprise at least one of memory resources and thread resources.

10. An electronic device, comprising:

a processor; and

a memory coupled to the processor and having instructions stored therein, wherein the instructions, when executed by the processor, cause the device to execute actions comprising:

determining a first dependency relationship between a plurality of computing tasks included in a to-be-processed computing job;

determining, based on the first dependency relationship and demands of the plurality of computing tasks for computing resources, a group of computing tasks for combination from the plurality of computing tasks; and

combining the group of computing tasks into a target computing task;

wherein determining the group of computing tasks from the plurality of computing tasks comprises:

determining, for each computing task of the plurality of computing tasks and based on the first dependency relationship, the number of computing tasks associated with at least one of an input or an output of a corresponding computing task;

determining a first group of candidate tasks from the plurality of computing tasks, wherein the number determined for each candidate task is less than a threshold number;

determining, based on demands of the first group of candidate tasks for the computing resources, a second group of candidate tasks for combination from the first group of candidate tasks; and

selecting the group of computing tasks from the second group of candidate tasks.

11. The device according to claim 10 , wherein the actions further comprise:

determining, based on the first dependency relationship, a second dependency relationship between the target computing task and computing tasks that are other than the group of computing tasks in the plurality of computing tasks.

12. The device according to claim 10 , wherein determining the second group of candidate tasks from the first group of candidate tasks comprises:

determining the second group of candidate tasks by removing a candidate task with a demand for the computing resources exceeding a first threshold from the first group of candidate tasks.

13. The device according to claim 10 , wherein selecting the group of computing tasks from the second group of candidate tasks comprises:

determining, based on the first dependency relationship, a first candidate task and a second candidate task in the second group of candidate tasks, wherein the first candidate task does not depend on other candidate tasks in the second group of candidate tasks, and the second candidate task depends on the first candidate task;

estimating a first demand of a first combined task that combines the first candidate task and the second candidate task for the computing resources; and

selecting, if it is determined that the estimated first demand is less than a second threshold, the first candidate task and the second candidate task as at least a part of the group of computing tasks.

14. The device according to claim 13 , further comprising:

estimating, if it is determined that the second group of candidate tasks includes a third candidate task depending on the second candidate task, a second demand of a second combined task that combines the first combined task and the third candidate task for the computing resources; and

adding, if it is determined that the estimated second demand is less than the second threshold, the third candidate task into the group of computing tasks.

15. The device according to claim 13 , further comprising:

determining, if it is determined that a third candidate task depending on the second candidate task is not included in the second group of candidate tasks, the first candidate task and the second candidate task as the group of computing tasks.

16. The device according to claim 10 , wherein determining the first dependency relationship comprises:

obtaining a computation graph representing the computing job, a vertex in the computation graph corresponding to a corresponding computing task among the plurality of computing tasks; and

determining the first dependency relationship from the computation graph.

17. The device according to claim 10 , wherein combining the group of computing tasks into the target computing task comprises:

obtaining a group of generation logics corresponding to the group of computing tasks, each generation logic in the group of generation logics being used for generating an executable instruction for a corresponding computing task; and

combining the group of generation logics into a target logic for generating an executable instruction for the target computing task.

18. The device according to claim 10 , wherein the computing resources comprise at least one of memory resources and thread resources.

19. A computer program product tangibly stored in a non-transitory computer-readable medium and comprising machine-executable instructions, wherein the machine-executable instructions, when executed, cause a machine to execute a method for processing a computing job, the method comprising:

determining a first dependency relationship between a plurality of computing tasks included in a to-be-processed computing job;

determining, based on the first dependency relationship and demands of the plurality of computing tasks for computing resources, a group of computing tasks for combination from the plurality of computing tasks; and

combining the group of computing tasks into a target computing task;

wherein determining the group of computing tasks from the plurality of computing tasks comprises:

determining, for each computing task of the plurality of computing tasks and based on the first dependency relationship, the number of computing tasks associated with at least one of an input or an output of a corresponding computing task;

determining a first group of candidate tasks from the plurality of computing tasks, wherein the number determined for each candidate task is less than a threshold number;

determining, based on demands of the first group of candidate tasks for the computing resources, a second group of candidate tasks for combination from the first group of candidate tasks; and

selecting the group of computing tasks from the second group of candidate tasks.

20. The computer program product according to claim 19 , wherein the method executed by the machine further comprises:

determining, based on the first dependency relationship, a second dependency relationship between the target computing task and computing tasks that are other than the group of computing tasks in the plurality of computing tasks.

Assignments (9)