Job scheduling method, device, and distributed system

1. A system comprising:

a central node including one or more first processors and first memory; and

a plurality of control nodes including at least a first control node and a second control node, the first control node including one or more second processors and second memory, and the second node including one or more third processors and third memory, wherein:

the central node assigns a plurality of tasks of a job to the plurality of control nodes respectively, the plurality of tasks including a first task and a second task, wherein the first task is an execution task in the job and is divided into a plurality of task instances, and the second task is another task dependent on the first task in the job;

the first control node scheduling the first task notifies the second control node scheduling the second task to obtain execution data generated by at least one task instance of the first task when the at least one task instance of the first task completes execution; and

the second control node obtains the execution data generated by the at least one task instance of the first task and assigns the execution data to each task instance of the second task, schedules to execute at least one task instance of the second task to process the execution data without waiting for a completion of the plurality of task instances of the first task, requests the first control node to schedule to re-execute a task instance, corresponding to the execution data version, of the first task when a required data version of a first task instance of the second task does not match an execution data version of the execution data, and obtains re-generated execution data version that is re-generated by the first control node.

2. The system of claim 1 , wherein the second control node further performs acts including:

scheduling to execute the first task instance of the second task when the re-generated execution data version matches the required data version of the first task instance.

3. The system of claim 1 , wherein the second control node further performs acts including:

sending a request to the first control node to re-execute the task instance, corresponding to the execution data version, of the first task until a currently re-generated execution data version matches the required data version of the first task instance, when the re-generated execution data version does not match the required data version of the first task instance.

4. The system of claim 3 , wherein the first control node further performs acts including:

after receiving the request from the second control node, scheduling to re-execute the task instance, corresponding to the execution data version, of the first task; and

notifying the second control node to obtain the re-generated execution data when the task instance, corresponding to the execution data version, of the first task completes re-execution.

5. The system of claim 1 , wherein the second control node further performs acts including:

sending a request to the first control node to schedule to re-execute a task instance, corresponding to the execution data version, of the first task when a second task instance of the second task fails to process the execution data;

obtains the re-generated execution data; and

schedules to re-execute the second task instance of the second task to process the re-generated execution data.

6. The system of claim 5 , wherein the first control node further performs acts including:

after receiving the request from the second control node to schedule to re-execute the task instance, corresponding to the execution data version, of the first task, scheduling to re-execute the task instance, corresponding to the execution data version, of the first task; and

notifying the second control node to obtain the re-generated execution data after the task instance, corresponding to the execution data version, of the first task completes re-execution.

7. The system of claim 5 , wherein the sending the request to the first control node to schedule includes:

requesting the first control node to schedule to re-execute the task instance, corresponding to the execution data version, of the first task by increasing the required data version.

8. A method comprising:

obtaining, by a second control node including one or more second processors and second memory, execution data generated by at least one task instance of a first task in a job, wherein the second control node schedules a second task in the job, the first task is scheduled by a first control node including one or more first processors and first memory, the first task is an execution task in the job and is divided into a plurality of task instances, the second task is another task dependent on the first task in the job;

assigning the execution data to each task instance of the second task;

scheduling to execute at least one task instance of the second task to process the execution data without waiting for a completion of the plurality of task instances of the first task;

when a required data version of a first task instance of the second task does not match an execution data version of the execution data, requesting the first control node to re-execute a task instance, corresponding to the execution data version, of the first task; and

obtaining re-generated execution data version that is re-generated by the first control node.

9. The method of claim 8 , wherein:

the job includes the first task and the second task; and

the first control and the second control node are connected to a central node including one or more third processors and third memory that assigns the first task to the first control node and the second task to the second control node.

10. The method of claim 8 , wherein scheduling to execute the at least one task instance of the second task to process the execution data includes:

scheduling a task instance whose required data version matches the execution data version to execute; and

processing the execution data.

11. The method of claim 8 , further comprising:

scheduling an execution of the first task instance when the re-generated execution data version matches the required data version of the first task instance.

12. The method of claim 8 , wherein requesting the first control node to re-execute the task instance, corresponding to the execution data version, of the first task comprises:

requesting the first control node to re-execute the task instance, corresponding to the execution data version, of the first task until a currently re-generated execution data version matches the required data version of the first task instance.

13. The method of claim 8 , wherein after scheduling to execute the at least one task instance of the second task to process the execution data, the method further comprises:

requesting the first control node to schedule to re-execute the task instance, corresponding to the execution data version, of the first task when the second control node fails to process the execution data at a second task instance of the second task;

obtaining the re-generated execution data from the first control node; and

scheduling to re-execute the second task instance of the second task and processing the re-generated execution data.

14. The method of claim 8 , wherein requesting the first control node to re-execute the task instance, corresponding to the execution data version, of the first task includes:

requesting the first control node to re-execute the task instance, corresponding to the execution data version, of the first task by increasing the required data version.

15. One or more computer readable media storing executable instructions that, when executed by a second control node, cause the second control node to perform acts comprising:

obtaining execution data generated by at least one task instance of a first task in a job, wherein the second control node schedules a second task in the job, the first task is scheduled by a first control node, the first task is an execution task in the job and is divided into a plurality of task instances, the second task is another task dependent on the first task in the job;

assigning the execution data to each task instance of the second task;

scheduling to execute at least one task instance of the second task to process the execution data without waiting for a completion of the plurality of task instances of the first task;

sending a request to the first control node to re-execute a task instance, corresponding to an execution data version, of the first task when a second task instance of the second task fails to process the execution data; and

obtaining re-generated execution data from the first control node.

16. One or more computer readable media of claim 15 , wherein scheduling to execute the at least one task instance of the second task to process the execution data comprises:

scheduling a task instance whose required data version matches the execution data version to execute; and

processing the execution data.

17. One or more computer readable media of claim 15 , the acts further comprising:

scheduling an execution of the task instance of the first task when the re-generated execution data version matches a required data version of the instance of the first task.

18. One or more computer readable media of claim 15 , wherein sending the request to the first control node to re-execute the task instance, corresponding to the execution data version, of the first task, comprises:

requesting the first control node to re-execute the task instance, corresponding to the execution data version, of the first task by increasing the required data version.

19. One or more computer readable media of claim 15 , the acts further comprising:

scheduling to re-execute the second task instance of the second task and processing the re-generated execution data.