THREAD TILING FOR MEMORY LATENCY REDUCTION

1 . A method for reducing memory latency utilizing thread tiling, comprising:

receiving a tensor input as an array of values, the tensor having a size including a first dimension, a second dimension, and a third dimension;

determining an operation type applied to the received tensor input;

splitting the received tensor input into a plurality of tiles based on: a dimension of the array of values, a number of single instruction multiple data (SIMD) processors, and an amount of available internal memory; and

processing each tile of the plurality of tiles on a SIMD processor.

2 . The method of claim 1 , further comprising:

determining a size of a kernel; and

splitting the received tensor further based on the determined size of the kernel.

3 . The method of claim 2 , further comprising:

configuring the SIMD processor to perform the determined operation between the kernel and each tile of the plurality of tiles.

4 . The method of claim 2 , wherein the size of the kernel includes: a value of the first dimension, a value of the second dimension, and a value of the third dimension.

5 . The method of claim 4 , further comprising:

selecting a first splitting scheme based on the largest value between the values of the first dimension, the second dimension and the third dimension of the input tensor; and

selecting a second splitting scheme based on the largest value between the values of the first dimension, the second dimension and the third dimension of a filter.

6 . The method of claim 5 , further comprising:

selecting the first splitting scheme in response to detecting the first splitting scheme is identical to the second splitting scheme.

7 . The method of claim 5 , further comprising:

selecting the first splitting scheme in response to detecting that the first splitting scheme is not identical to the second splitting scheme and processor optimization is selected.

8 . The method of claim 5 , further comprising:

selecting the second splitting scheme in response to detecting that the first splitting scheme is not identical to the second splitting scheme, and memory optimization is selected.

9 . The method of claim 1 , further comprising:

splitting the tensor input into a plurality of slices based on the first dimension and the second dimension.

10 . The method of claim 9 , further comprising:

splitting each slice of the plurality of slices into a plurality of fibers based on the third dimension.

11 . The method of claim 1 , further comprising:

splitting the tensor input into a plurality of slices based on the second dimension and the third dimension.

12 . The method of claim 11 , further comprising:

splitting each slice of the plurality of slices into a plurality of fibers based on the third dimension.

13 . The method of claim 1 , further comprising:

splitting the tensor input further based on the largest value of: the first dimension, the second dimension, and the third dimension.

14 . The method of claim 1 , further comprising:

detecting a symmetry in the tensor input; and

further splitting the tensor input into a plurality of tiles based on the detected symmetry.

15 . The method of claim 1 , further comprising:

splitting the received tensor input into a plurality of tiles further based on any one of: a size of an available register, a number of currently available registers, a length of an SIMD lane, and a combination thereof.

16 . A non-transitory computer-readable medium storing a set of instructions for reducing memory latency utilizing thread tiling, the set of instructions comprising:

one or more instructions that, when executed by one or more processors of a device, cause the device to:

receive a tensor input as an array of values, the tensor having a size including a first dimension, a second dimension, and a third dimension;

determine an operation type applied to the received tensor input;

split the received tensor input into a plurality of tiles based on: a dimension of the array of values, a number of single instruction multiple data (SIMD) processors, and an amount of available internal memory; and

process each tile of the plurality of tiles on a SIMD processor.

17 . A system for reducing memory latency utilizing thread tiling comprising:

a processing circuitry;

a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to:

receive a tensor input as an array of values, the tensor having a size including a first dimension, a second dimension, and a third dimension;

determine an operation type applied to the received tensor input;

split the received tensor input into a plurality of tiles based on: a dimension of the array of values, a number of single instruction multiple data (SIMD) processors, and an amount of available internal memory; and

process each tile of the plurality of tiles on a SIMD processor.

18 . The system of claim 17 , wherein the memory contains further instructions which when executed by the processing circuitry further configure the system to:

determine a size of a kernel; and

split the received tensor further based on the determined size of the kernel.

19 . The system of claim 18 , wherein the memory contains further instructions which when executed by the processing circuitry further configure the system to:

configure the SIMD processor to perform the determined operation between the kernel and each tile of the plurality of tiles.

20 . The system of claim 18 , wherein the size of the kernel includes:

a value of the first dimension, a value of the second dimension, and a value of the third dimension.

21 . The system of claim 20 , wherein the memory contains further instructions which when executed by the processing circuitry further configure the system to:

select a first splitting scheme based on the largest value between the values of the first dimension, the second dimension and the third dimension of the input tensor; and

select a second splitting scheme based on the largest value between the values of the first dimension, the second dimension and the third dimension of a filter.

22 . The system of claim 21 , wherein the memory contains further instructions which when executed by the processing circuitry further configure the system to:

select the first splitting scheme in response to detecting the first splitting scheme is identical to the second splitting scheme.

23 . The system of claim 21 , wherein the memory contains further instructions which when executed by the processing circuitry further configure the system to:

select the first splitting scheme in response to detecting that the first splitting scheme is not identical to the second splitting scheme and processor optimization is selected.

24 . The system of claim 21 , wherein the memory contains further instructions which when executed by the processing circuitry further configure the system to:

select the second splitting scheme in response to detecting that the first splitting scheme is not identical to the second splitting scheme, and memory optimization is selected.

25 . The system of claim 17 , wherein the memory contains further instructions which when executed by the processing circuitry further configure the system to:

split the tensor input into a plurality of slices based on the first dimension and the second dimension.

26 . The system of claim 25 , wherein the memory contains further instructions which when executed by the processing circuitry further configure the system to:

split each slice of the plurality of slices into a plurality of fibers based on the third dimension.

27 . The system of claim 17 , wherein the memory contains further instructions which when executed by the processing circuitry further configure the system to:

split the tensor input into a plurality of slices based on the second dimension and the third dimension.

28 . The system of claim 27 , wherein the memory contains further instructions which when executed by the processing circuitry further configure the system to:

split each slice of the plurality of slices into a plurality of fibers based on the third dimension.

29 . The system of claim 17 , wherein the memory contains further instructions which when executed by the processing circuitry further configure the system to:

split the tensor input further based on the largest value of: the first dimension, the second dimension, and the third dimension.

30 . The system of claim 17 , wherein the memory contains further instructions which when executed by the processing circuitry further configure the system to:

detect a symmetry in the tensor input; and

further split the tensor input into a plurality of tiles based on the detected symmetry.

31 . The system of claim 17 , wherein the memory contains further instructions which when executed by the processing circuitry further configure the system to:

split the received tensor input into a plurality of tiles further based on any one of: a size of an available register, a number of currently available registers, a length of an SIMD lane, and a combination thereof.