TECHNIQUES FOR THREAD REDUCTION IN PROCESSING TENSORS UTILIZING SPARSITY DETECTION
A system and method reduces thread spawning in processing tensor inputs. The method includes generating a result of a block of a tensor input with a predefined kernel on a processing circuitry, the processing circuitry configured to process in parallel a plurality of threads; performing a depth test on the generated result; generating for an indicator bit a first indicator bit value based on the depth test, the first indicator bit value indicating that the result is sparse; generating for the indicator bit a second indicator bit value based on the depth test, the second indicator bit value indicating that the result is not sparse; and spawning a thread for processing the result and a second predefined kernel, based on the indicator bit having the second indicator bit value.
1 . A method for thread reduction in tensor processing, comprising:
generating a result of a block of a tensor input with a predefined kernel on a processing circuitry, the processing circuitry configured to process in parallel a plurality of threads;
performing a depth test on the generated result;
generating for an indicator bit a first indicator bit value based on the depth test, the first indicator bit value indicating that the result is sparse;
generating for the indicator bit a second indicator bit value based on the depth test, the second indicator bit value indicating that the result is not sparse; and
spawning a thread for processing the result and a second predefined kernel, based on the indicator bit having the second indicator bit value.
2 . The method of claim 1 , further comprising:
storing the result and a value of the indicator bit in a memory.
3 . The method of claim 1 , further comprising:
generating for the indicator bit a value indicating that the result is sparse, in response to determining that the depth test indicates that sparsity of the block is within a threshold value.
4 . The method of claim 3 , wherein the threshold is any one of: dynamic, static, and adaptive.
5 . The method of claim 1 , further comprising:
performing a second depth test on a plurality of results, the plurality of results including the generated result.
6 . The method of claim 5 , further comprising:
generating for the indicator bit the first indicator bit value based on the second depth test, the first indicator bit value indicating that the result is sparse.
7 . The method of claim 1 , wherein the generated result is a block representing a plurality of pixels.
8 . The method of claim 1 , further comprising:
processing the thread on a core of the processing circuitry, wherein the processing circuitry is a multi-core processing circuitry; and
generating a second result based on an output of the core.
9 . The method of claim 8 , further comprising:
generating the second result further based on a plurality of zero values, in response to determining that the indicator bit has the first indicator bit value.
10 . The method of claim 1 , further comprising:
performing structured pruning of the predefined kernel.
11 . The method of claim 1 , further comprising:
performing unstructured pruning of the predefined kernel.
12 . The method of claim 1 , further comprising:
generating a value for the indicator bit based on a weight associated with the predefined kernel.
13 . The method of claim 1 , further comprising:
quantizing a value of the block, wherein the block includes a plurality of values.
14 . The method of claim 13 , wherein the value is stored as a floating point value, and the quantized value is stored as an integer.
15 . A non-transitory computer readable medium having stored thereon instructions for causing a processing circuitry to execute a process, the process comprising:
generating a result of a block of a tensor input with a predefined kernel on a processing circuitry, the processing circuitry configured to process in parallel a plurality of threads;
performing a depth test on the generated result;
generating for an indicator bit a first indicator bit value based on the depth test, the first indicator bit value indicating that the result is sparse;
generating for the indicator bit a second indicator bit value based on the depth test, the second indicator bit value indicating that the result is not sparse; and
spawning a thread for processing the result and a second predefined kernel, based on the indicator bit having the second indicator bit value.
16 . A system for thread reduction in tensor processing, comprising:
a processing circuitry; and
a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to:
generate a result of a block of a tensor input with a predefined kernel on a processing circuitry, the processing circuitry configured to process in parallel a plurality of threads;
perform a depth test on the generated result;
generate for an indicator bit a first indicator bit value based on the depth test, the first indicator bit value indicating that the result is sparse;
generate for the indicator bit a second indicator bit value based on the depth test, the second indicator bit value indicating that the result is not sparse; and
spawn a thread for processing the result and a second predefined kernel, based on the indicator bit having the second indicator bit value.
17 . The method of claim 16 , further comprising:
storing the result and a value of the indicator bit in a memory.
18 . The method of claim 16 , further comprising:
generating for the indicator pit a value indicating that the result is sparse, in response to determining that the depth test indicates that sparsity of the block is within a threshold value.
19 . The method of claim 18 , wherein the threshold is any one of: dynamic, static, and adaptive.
20 . The method of claim 16 , further comprising:
performing a second depth test on a plurality of results, the plurality of results including the generated result.
21 . The method of claim 20 , further comprising:
generating for the indicator bit the first indicator bit value based on the second depth test, the first indicator bit value indicating that the result is sparse.
22 . The method of claim 16 , wherein the generated result is a block representing a plurality of pixels.
23 . The method of claim 16 , further comprising:
processing the thread on a core of the processing circuitry, wherein the processing circuitry is a multi-core processing circuitry; and
generating a second result based on an output of the core.
24 . The method of claim 23 , further comprising:
generating the second result further based on a plurality of zero values, in response to determining that the indicator bit has the first indicator bit value.
25 . The method of claim 16 , further comprising:
performing structured pruning of the predefined kernel.
26 . The method of claim 16 , further comprising:
performing unstructured pruning of the predefined kernel.
27 . The method of claim 16 , further comprising:
generating a value for the indicator bit based on a weight associated with the predefined kernel.
28 . The method of claim 16 , further comprising:
quantizing a value of the block, wherein the block includes a plurality of values.
29 . The method of claim 28 , wherein the value is stored as a floating point value, and the quantized value is stored as an integer.