Application programming interface to wait on matrix multiply-accumulate
Apparatuses, systems, and techniques to perform computational operations in response to one or more compute uniform device architecture (CUDA) programs. In at least one embodiment, one or more computational operations are to cause one or more other computational operations to wait until a portion of matrix multiply-accumulate (MMA) operations have been performed.
1 . One or more processors, comprising:
circuitry to:
determine, using one or more parameters of a call to an instruction, a threshold number of waiting matrix multiply-accumulate (MMA) operations to be performed by one or more threads;
determine, using the threshold number of waiting MMA operations, whether a current number of waiting MMA operations satisfies the threshold; and
cause, based on whether the current number of waiting MMA operations satisfies the threshold, one or more second threads to pause a performance of one or more MMA operations.
2 . The one or more processors of claim 1 , wherein the instruction is to cause one or more threads comprising one or more instructions to wait until one or more waiting MMA operations have been performed.
3 . The one or more processors of claim 1 , wherein the instruction is to cause one or more threads comprising one or more instructions to perform one or more other instructions and, in response to the instruction, wait until one or more waiting MMA operations have been performed.
4 . The one or more processors of claim 1 , wherein the instruction is a wait instruction and the waiting MMA operations are one or more groups of asynchronous MMA operations to be performed.
5 . The one or more processors of claim 1 , wherein the call to an instruction is an application programming interface (API) call.
6 . The one or more processors of claim 1 , wherein the waiting MMA operations have been performed if one or more results of said waiting MMA operations is stored in memory.
7 . The one or more processors of claim 1 , wherein the one or more parameters comprise a constant integer data value corresponding to the threshold number of waiting MMA operations.
8 . The one or more processors of claim 1 , wherein the one or more processors are graphics processing units (GPUs).
9 . A system comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the system to:
determine, using one or more parameters of a call to an instruction, a threshold number of waiting matrix multiply-accumulate (MMA) operations to be performed by one or more threads;
determine, using the threshold number of waiting MMA operations, whether a current number of waiting MMA operations satisfies the threshold; and
cause, based on whether the current number of waiting MMA operations satisfies the threshold, one or more second threads to pause a performance of one or more MMA operations.
10 . The system of claim 9 , wherein the instruction is to cause one or more threads comprising one or more instructions to wait until the current number of waiting MMA operations satisfies the threshold.
11 . The system of claim 9 , wherein the instruction is to cause one or more threads comprising one or more instructions to wait until one or more waiting MMA operations have been performed.
12 . The system of claim 9 , wherein the call to an instruction is an application programming interface (API) call.
13 . The system of claim 9 , wherein the instruction is to cause one or more threads comprising one or more instructions to perform one or more other instructions and, in response to the instruction, wait until one or more waiting MMA operations have been performed.
14 . The system of claim 9 , wherein the one or more processors are graphics processing units (GPUs).
15 . A method comprising:
receiving an instruction call indicating one or more parameters;
determining a threshold number of waiting matrix multiply-accumulate (MMA) operations to be performed by one or more threads indicated by the one or more parameters of the instruction call;
determining, using the threshold number of waiting MMA operations, whether a current number of waiting MMA operations satisfies the threshold; and
causing, based on whether the current number of waiting MMA operations satisfies the threshold, one or more second threads to pause a performance of one or more waiting MMA operations.
16 . The method of claim 15 , further comprising causing, in response to the instruction call, one or more threads comprising one or more instructions to wait until the current number of waiting MMA operations satisfies the threshold.
17 . The method of claim 15 , further comprising causing, in response to the instruction call, one or more threads comprising one or more instructions to wait until a threshold quantity of groupings of the one or more MMA operations have been performed.
18 . The method of claim 15 , further comprising causing, in response to the instruction call, one or more threads comprising one or more instructions to perform one or more other instructions and, in response to the instruction, to wait until the one or more MMA operations have been performed.
19 . The method of claim 15 , wherein the one or more MMA operations are to be asynchronously performed by one or more accelerators of one or more graphics processing units (GPUs).
20 . The method of claim 15 , wherein the instruction call is an application programming interface (API) call.