Allreduce enhanced direct memory access functionality
View Patent ↗Systems, apparatuses, and methods for performing an allreduce operation on an enhanced direct memory access (DMA) engine are disclosed. A system implements a machine learning application which includes a first kernel and a second kernel. The first kernel corresponds to a first portion of a machine learning model while the second kernel corresponds to a second portion of the machine learning model. The first kernel is invoked on a plurality of compute units and the second kernel is converted into commands executable by an enhanced DMA engine to perform a collective communication operation. The first kernel is executed on the plurality of compute units in parallel with the enhanced DMA engine executing the commands for performing the collective communication operation. As a result, the allreduce operation may be executed in parallel on the enhanced DMA engine to the compute units.
1. An enhanced direct memory access (DMA) engine comprising:
an arithmetic logic unit (ALU); and
control logic comprising circuitry configured to:
detect a command that identifies an operation, wherein the command corresponds to a kernel converted into one or more commands executable by the enhanced DMA;
perform a DMA transfer operation, comprising at least a read operation to a memory to load data into the enhanced DMA engine, responsive to an operation flag in the command having a first value; and
responsive to the operation flag in the command has having a second value:
perform one or more read operations to the memory to load first data comprising multiple data inputs into the enhanced DMA engine;
cause the ALU to combine the multiple data inputs into a single data output; and
convey the single data output.
2. The enhanced DMA engine as recited in claim 1 , wherein when the operation flag has the second value, the command includes a field specifying the one or more read operations to be performed by the ALU.
3. The enhanced DMA engine as recited in claim 1 , wherein the control logic operates in a first mode when the operation flag has the first value, and wherein the control logic operates in a second mode when the operation flag has the second value.
4. The enhanced DMA engine as recited in claim 1 , wherein the kernel targets a plurality of compute units of a processor.
5. The enhanced DMA engine as recited in claim 1 , wherein the enhanced DMA engine is configured to broadcast the single data output.
6. The enhanced DMA engine as recited in claim 1 , wherein the command includes a data-type flag which specifies a data format of operands being operated on by the ALU.
7. The enhanced DMA engine as recited in claim 6 , wherein the data-type flag specifies one or more of bitfield, signed integer, unsigned integer, characters, standard floating-point, custom floating point, fixed-point fractions, bit-width, and/or combinations of multiple values.
8. A method comprising:
detecting, by circuitry of an enhanced direct memory access (DMA) engine, a command that identifies an operation, wherein the command corresponds to a kernel converted into one or more enhanced DMA commands;
performing a DMA transfer operation, comprising at least a read operation to a memory to load data into the enhanced DMA engine, responsive to an operation flag in the command having a first value;
responsive to the operation flag in the command having a second value, the circuitry of the enhanced DMA engine:
performing one or more read operations to load first data comprising multiple data inputs into the enhanced DMA engine;
causing an arithmetic logic unit (ALU) to combine the multiple data inputs into a single data output; and
conveying the single data output.
9. The method as recited in claim 8 , wherein when the operation flag has the second value, the command includes a field specifying the one or more read operations to be performed by the ALU.
10. The method as recited in claim 8 , further comprising:
operating in a first mode when the operation flag has the first value; and
operating in a second mode when the operation flag has the second value.
11. The method as recited in claim 8 , wherein the command corresponds to a collective communication phase of a machine learning model.
12. The method as recited in claim 8 , further comprising broadcasting the single data output.
13. The method as recited in claim 8 , wherein the command includes a data-type flag which specifies a data format of operands being operated on by the ALU.
14. The method as recited in claim 13 , wherein the data-type flag specifies one or more of bitfield, signed integer, unsigned integer, characters, standard floating-point, custom floating point, fixed-point fractions, bit-width, and/or combinations of multiple values.
15. A system comprising:
a plurality of compute units comprising circuitry configured to execute a given workload; and
an enhanced direct memory access (DMA) engine comprising circuitry configured to:
detect a command targeting the enhanced DMA engine, wherein the command corresponds to a kernel converted into one or more commands executable by the enhanced DMA;
perform a DMA transfer operation, comprising at least a read operation to a memory to load data into the enhanced DMA engine, responsive to an operation flag in the command having a first value; and
responsive to the operation flag in the command having a second value:
perform one or more read operations to the memory to load first data comprising multiple data inputs into the enhanced DMA engine;
combine the multiple data inputs into a single data output; and
convey the single data output.
16. The system as recited in claim 15 , wherein when the operation flag has the second value, the command includes a field specifying the one or more read operations to be performed.
17. The system as recited in claim 16 , wherein the enhanced DMA engine operates in a first mode when the operation flag has the first value, and wherein the enhanced DMA engine operates in a second mode when the operation flag has the second value.
18. The system as recited in claim 17 , wherein the enhanced DMA engine causes an ALU to enter a reduced-power state when operating in the first mode.
19. The system as recited in claim 15 , wherein the enhanced DMA engine is configured to broadcast the single data output.
20. The system as recited in claim 15 , wherein the command includes a data-type flag which specifies a data format of operands being operated on by the enhanced DMA engine.