Method, electronic device and computer program product for processing task
Embodiments of the present disclosure provide a method, an electronic device and a computer program product for processing a task. The method comprises: obtaining a first group of processing results generated from processing, by a first group of processing resources of a first device, a first group of sub-tasks in the task; performing a first AllReduce operation on the first group of processing results to obtain a first AllReduce result; obtaining a second AllReduce result from a second device, the second AllReduce result being obtained by performing a second AllReduce operation on a second group of processing results generated from processing, by a second group of processing resources of the second device, a second group of sub-tasks in the task; and performing a third AllReduce operation on the first AllReduce result and the second AllReduce result to obtain a processing result of the task.
1. A method of processing a task, comprising:
initiating, at an application of a user device, a task for processing, the task being divided into a plurality of sub-tasks;
responsive to each sub-task of the plurality of sub-tasks being assigned to a respective processing resource of a corresponding group of processing resources of the user device,
receiving, at a first group of processing resources of a first device from one or more respective processing resources of the corresponding group of processing resources of the user device, a first group of sub-tasks in the task;
receiving, at a second group of processing resources of a second device from one or more other respective processing resources of the corresponding group of processing resources of the user device, a second group of sub-tasks in the task; and
receiving, at a third group of processing resources of a third device from one or more remaining respective processing resources of the corresponding group of processing resources of the user device, a third group of sub-tasks in the task;
obtaining a first group of processing results generated from processing, by the first group of processing resources of the first device, the first group of sub-tasks in the task;
performing, by the first device, a first reduce operation to aggregate the first group of processing results to obtain a first reduce result;
obtaining a second reduce result from the second device, the second reduce result being obtained by performing a second reduce operation to aggregate a second group of processing results generated from processing, by the second group of processing resources of the second device, the second group of sub-tasks in the task;
obtaining at least one processing result generated from processing, by at least one processing resource of the third group of processing resources of the third device, the third group of sub-tasks in the task;
performing a third reduce operation on the first reduce result, the second reduce result, and the processing result of the at least one processing resource of the third group of processing resources of the third device, to obtain a processing result of the task; and
providing the processing result of the task to one or more processing resources of at least one of the first group of processing resources of the first device, the second group of processing resources of the second device and the third group of processing resources of the third device for further processing;
wherein each of the processing resources in the group of processing resources of the user device has a one-to-one correspondence with a respective processing resource in the processing resources of the first group of processing resources of the first device, the second group of processing resources of the second device and the third group of processing resources of the third device.
2. The method of claim 1 , wherein the first reduce operation, the second reduce operation and the third reduce operation comprise a first AllReduce operation, a second AllReduce operation and a third AllReduce operation, respectively, and are of a same type.
3. The method of claim 1 , wherein the first reduce operation, the second reduce operation and the third reduce operation comprise at least one of the following types of operations:
summation,
averaging,
taking a maximum value, and
taking a minimum value.
4. The method of claim 1 , wherein the processing resources comprise dedicated processing resources.
5. The method of claim 4 , wherein the dedicated processing resources comprise at least one of a graphics processing unit (GPU), a field programmable gate array (FPGA) and an artificial intelligence processor (AI processor).
6. The method of claim 1 , further comprising:
providing the processing result of the task to the first group of processing resources of the first device.
7. An electronic device, comprising:
a processor; and
a memory storing computer program instructions, the processor executing the computer program instructions in the memory to control the device to perform actions comprising:
initiating, at an application of a user device, a task for processing, the task being divided into a plurality of sub-tasks;
responsive to each sub-task of the plurality of sub-tasks being assigned to a respective processing resource of a corresponding group of processing resources of the user device,
receiving, at a first group of processing resources of a first device from one or more respective processing resources of the corresponding group of processing resources of the user device, a first group of sub-tasks in the task;
receiving, at a second group of processing resources of a second device from one or more other respective processing resources of the corresponding group of processing resources of the user device, a second group of sub-tasks in the task; and
receiving, at a third group of processing resources of a third device from one or more remaining respective processing resources of the corresponding group of processing resources of the user device, a third group of sub-tasks in the task;
obtaining a first group of processing results generated from processing, by the first group of processing resources of the first device, the first group of sub-tasks in the task;
performing, by the first device, a first reduce operation to aggregate the first group of processing results to obtain a first reduce result;
obtaining a second reduce result from the second device, the second reduce result being obtained by performing a second reduce operation to aggregate a second group of processing results generated from processing, by the second group of processing resources of the second device, the second group of sub-tasks in the task;
obtaining at least one processing result generated from processing, by at least one processing resource of the third group of processing resources of the third device, the third group of sub-tasks in the task;
performing a third reduce operation on the first reduce result, the second reduce result, and the processing result of the at least one processing resource of the third group of processing resources of the third device, to obtain a processing result of the task; and
providing the processing result of the task to one or more processing resources of at least one of the first group of processing resources of the first device, the second group of processing resources of the second device and the third group of processing resources of the third device for further processing;
wherein each of the processing resources in the group of processing resources of the user device has a one-to-one correspondence with a respective processing resource in the processing resources of the first group of processing resources of the first device, the second group of processing resources of the second device and the third group of processing resources of the third device.
8. The electronic device of claim 7 , wherein the first reduce operation, the second reduce operation and the third reduce operation comprise a first AllReduce operation, a second AllReduce operation and a third AllReduce operation, respectively, and are of a same type.
9. The electronic device of claim 7 , wherein the first reduce operation, the second reduce operation and the third reduce operation comprise at least one of the following types of operations:
summation,
averaging,
taking a maximum value, and
taking a minimum value.
10. The electronic device of claim 7 , wherein the processing resources comprise dedicated processing resources.
11. The electronic device of claim 10 , wherein the dedicated processing resources comprise at least one of a graphics processing unit (GPU), a field programmable gate array (FPGA) and an artificial intelligence processor (AI processor).
12. The electronic device of claim 7 , wherein the actions further comprise:
providing the processing result of the task to the first group of processing resources of the first device.
13. A computer program product being tangibly stored on a non-transitory computer storage medium and comprising machine-executable instructions which, when executed by a device, cause the device to perform actions comprising:
initiating, at an application of a user device, a task for processing, the task being divided into a plurality of sub-tasks;
responsive to each sub-task of the plurality of sub-tasks being assigned to a respective processing resource of a corresponding group of processing resources of the user device,
receiving, at a first group of processing resources of a first device from one or more respective processing resources of the corresponding group of processing resources of the user device, a first group of sub-tasks in the task;
receiving, at a second group of processing resources of a second device from one or more other respective processing resources of the corresponding group of processing resources of the user device, a second group of sub-tasks in the task; and
receiving, at a third group of processing resources of a third device from one or more remaining respective processing resources of the corresponding group of processing resources of the user device, a third group of sub-tasks in the task;
obtaining a first group of processing results generated from processing, by the first group of processing resources of the first device, the first group of sub-tasks in the task;
performing, by the first device, a first reduce operation to aggregate the first group of processing results to obtain a first reduce result;
obtaining a second reduce result from the second device, the second reduce result being obtained by performing a second reduce operation to aggregate a second group of processing results generated from processing, by the second group of processing resources of the second device, the second group of sub-tasks in the task;
obtaining at least one processing result generated from processing, by at least one processing resource of the third group of processing resources of the third device, the third group of sub-tasks in the task;
performing a third reduce operation on the first reduce result, the second reduce result, and the processing result of the at least one processing resource of the third group of processing resources of the third device, to obtain a processing result of the task; and
providing the processing result of the task to one or more processing resources of at least one of the first group of processing resources of the first device, the second group of processing resources of the second device and the third group of processing resources of the third device for further processing;
wherein each of the processing resources in the group of processing resources of the user device has a one-to-one correspondence with a respective processing resource in the processing resources of the first group of processing resources of the first device, the second group of processing resources of the second device and the third group of processing resources of the third device.
14. The computer program product of claim 13 , wherein the first reduce operation, the second reduce operation and the third reduce operation comprise a first AllReduce operation, a second AllReduce operation and a third AllReduce operation, respectively, and are of a same type.
15. The computer program product of claim 13 , wherein the first reduce operation, the second reduce operation and the third reduce operation comprise at least one of the following types of operations:
summation,
averaging,
taking a maximum value, and
taking a minimum value.
16. The computer program product of claim 13 , wherein the processing resources comprise dedicated processing resources.
17. The computer program product of claim 16 , wherein the dedicated processing resources comprise at least one of a graphics processing unit (GPU), a field programmable gate array (FPGA) and an artificial intelligence processor (AI processor).
18. The computer program product of claim 13 , wherein the machine-executable instructions which, when executed by the device, cause the device to further perform an action comprising:
providing the processing result of the task to the first group of processing resources of the first device.
19. The computer program product of claim 13 , wherein the machine-executable instructions which, when executed by the device, cause the device to further perform an action comprising:
providing the processing result of the task to the second group of processing resources of the second device.
20. The computer program product of claim 13 , wherein the machine-executable instructions which, when executed by the device, cause the device to further perform an action comprising:
providing the processing result of the task to the third group of processing resources of the third device.