Method and apparatus for utilizing external neural processor from graphics processor

1 . An apparatus comprising:

a common memory unit;

a first processing engine coupled to the common memory unit, wherein the first processing engine is configured to access a plurality of input fibers of an input tensor and a plurality of kernel fibers of a kernel tensor from the common memory unit; and

a second processing engine coupled to the common memory unit, wherein the first processing engine is further configured to send the plurality of input fibers of the input tensor and the plurality of kernel fibers of the kernel tensor to the second processing engine and wherein the second processing engine is configured to generate a plurality of output fibers of an output tensor based on the plurality of input fibers of the input tensor and on the plurality of kernel fibers of the kernel tensor, wherein the plurality of input fibers, the plurality of kernel fibers and the plurality of output fibers each have a single variable dimensional index; and wherein each of the plurality of input fibers is a subset of the input tensor with all indices but one fixed, wherein each of the plurality of kernel fibers is a subset of the kernel tensor with all indices but one fixed, and wherein each of the plurality of output fibers is a subset of the output tensor with all indices but one fixed.

2 . The apparatus of claim 1 , wherein the second processing engine is further configured to be accessed directly by the first processing engine to support a workload or a wave-level execution structure from the first processing engine.

3 . The apparatus of claim 2 , wherein the workload is a sum of all work tasks of the first processing engine.

4 . The apparatus of claim 2 , wherein the wave-level execution structure is a data structure which operates on the plurality of input fibers of the input tensor.

5 . The apparatus of claim 2 , wherein the wave-level execution structure is a data structure which operates on the plurality of kernel fibers of the kernel tensor.

6 . The apparatus of claim 5 , wherein the wave-level execution structure is the data structure which operates on the plurality of input fibers of the input tensor.

7 . The apparatus of claim 1 , wherein the second processing engine is further configured to execute one or more heterogeneous operations on the plurality of input fibers of the input tensor and the plurality of kernel fibers of the kernel tensor.

8 . The apparatus of claim 7 , wherein the one or more heterogeneous operations include matrix multiplication instructions received from the first processing engine.

9 . The apparatus of claim 7 , wherein the one or more heterogeneous operations include convolution instructions received from the first processing engine.

10 . The apparatus of claim 7 , wherein the one or more heterogeneous operations uses parallel tensor processing.

11 . The apparatus of claim 10 , wherein the one or more heterogeneous operations uses at least one wave-level parallel operation.

12 . The apparatus of claim 11 , wherein the at least one wave-level parallel operation executes on the plurality of input fibers and the plurality of kernel fibers in parallel.

13 . The apparatus of claim 11 , wherein the plurality of input fibers of the input tensor comprises an input wave.

14 . The apparatus of claim 13 , wherein the plurality of kernel fibers of the kernel tensor comprises a kernel wave.

15 . The apparatus of claim 14 , wherein the at least one wave-level parallel operation executes on the input wave and the kernel wave in parallel.

16 . A method for implementing concurrent tensor processing, the method comprising:

accessing a plurality of input fibers of an input tensor from a common memory unit;

accessing a plurality of kernel fibers of a kernel tensor from the common memory unit;

sending the plurality of input fibers of the input tensor to a neural processing unit (NPU);

sending the plurality of kernel fibers of the kernel tensor to the NPU; and

retrieving a plurality of output fibers of an output tensor from the common memory unit, wherein the plurality of output fibers of the output tensor is generated by the NPU based on the plurality of input fibers of the input tensor and on the plurality of kernel fibers of the kernel tensor, wherein the plurality of input fibers, the plurality of kernel fibers and the plurality of output fibers each have a single variable dimensional index; and wherein each of the plurality of input fibers is a subset of the input tensor with all indices but one fixed, wherein each of the plurality of kernel fibers is a subset of the kernel tensor with all indices but one fixed, and wherein each of the plurality of output fibers is a subset of the output tensor with all indices but one fixed.

17 . The method of claim 16 , further comprising sending a synchronization request message to the NPU.

18 . The method of claim 17 , wherein the synchronization request message is based on a polling synchronization scheme or on a hardware interrupt synchronization scheme.

19 . The method of claim 17 , further comprising receiving a synchronization response message from the NPU.

20 . The method of claim 19 , wherein the synchronization response message is based on a polling synchronization scheme or on a hardware interrupt synchronization scheme.

21 . The method of claim 16 , further comprising configuring the NPU to be accessed directly by a processing engine to support a workload or a wave-level execution structure from the processing engine.

22 . The method of claim 16 , wherein the plurality of input fibers of the input tensor comprises a plurality of input waves and the plurality of kernel fibers of the kernel tensor comprises a plurality of kernel waves.

23 . An apparatus for implementing concurrent tensor processing, the apparatus comprising:

means for accessing a plurality of input fibers of an input tensor from a common memory unit;

means for accessing a plurality of kernel fibers of a kernel tensor from the common memory unit;

means for sending the plurality of input fibers of the input tensor to a neural processing unit (NPU);

means for sending the plurality of kernel fibers of the kernel tensor to the NPU; and

means for retrieving a plurality of output fibers of an output tensor from the common memory unit, wherein the plurality of output fibers of the output tensor is generated by the NPU based on the plurality of input fibers of the input tensor and on the plurality of kernel fibers of the kernel tensor, wherein the plurality of input fibers, the plurality of kernel fibers and the plurality of output fibers each have a single variable dimensional index; and wherein each of the plurality of input fibers is a subset of the input tensor with all indices but one fixed, wherein each of the plurality of kernel fibers is a subset of the kernel tensor with all indices but one fixed, and wherein each of the plurality of output fibers is a subset of the output tensor with all indices but one fixed.

24 . The apparatus of claim 23 , further comprising means for configuring the NPU to be accessed directly by a processing engine to support a workload or a wave-level execution structure from the processing engine.

25 . A non-transitory computer-readable medium storing computer executable code, operable on a device comprising at least one processor and at least one memory coupled to the at least one processor, wherein the at least one processor is configured to implement concurrent tensor processing, the computer executable code comprising:

instructions for causing a computer to access a plurality of input fibers of an input tensor from a common memory unit;

instructions for causing the computer to access a plurality of kernel fibers of a kernel tensor from the common memory unit;

instructions for causing the computer to send the plurality of input fibers of the input tensor to a neural processing unit (NPU);

instructions for causing the computer to send the plurality of kernel fibers of the kernel tensor to the NPU; and

instructions for causing the computer to retrieve a plurality of output fibers of an output tensor from the common memory unit, wherein the plurality of output fibers of the output tensor is generated by the NPU based on the plurality of input fibers of the input tensor and on the plurality of kernel fibers of the kernel tensor, wherein the plurality of input fibers, the plurality of kernel fibers and the plurality of output fibers each have a single variable dimensional index; and wherein each of the plurality of input fibers is a subset of the input tensor with all indices but one fixed, wherein each of the plurality of kernel fibers is a subset of the kernel tensor with all indices but one fixed, and wherein each of the plurality of output fibers is a subset of the output tensor with all indices but one fixed.

26 . The non-transitory computer-readable medium of claim 25 , further comprising instructions for causing the computer to configure the NPU to be accessed directly by a processing engine to support a workload or a wave-level execution structure from the processing engine.