Processing core with operation suppression based on contribution estimate
Processing cores with the ability to suppress operations based on a contribution estimate for those operations for purposes of increasing the overall performance of the core are disclosed. Associated methods that can be conducted by such processing cores are also disclosed. One such method includes generating a reference value for a composite computation. A complete execution of the composite computation generates a precise output and requires execution of a set of component computations. The method also includes generating a component computation approximation. The method also includes evaluating the component computation approximation with the reference value. The method also includes executing a partial execution of the composite computation using the component computation approximation to produce an estimated output. The method also includes suppressing the component computation, while executing the partial execution, based on the evaluation of the component computation approximation with the reference value.
1 . A computer-implemented method, comprising:
generating a contribution estimate for a component computation using an operand for the component computation, wherein the component computation is part of a composite computation; and
suppressing, based on the contribution estimate and during execution of a partial execution of the composite computation, the component computation;
wherein the suppressing is performed by a suppression logic block and comprises dropping the component computation from the composite computation by selectively providing operands from a set of operands to a math unit.
2 . The computer-implemented method of claim 1 , further comprising:
determining, based on the contribution estimate, that suppressing the component computation results in a decrease in computational resources consumed attributable to the partial execution of the composite computation that outweighs a decrease in accuracy of an output from the partial execution of the composite computation.
3 . The computer-implemented method of claim 1 , wherein:
the partial execution of the composite computation generates a first result for the composite computation and requires a first amount of computational resources;
a complete execution of the composite computation generates a second result for the composite computation and requires a second amount of computational resources;
the first result is not equal to the second result; and
the first amount is less than the second amount.
4 . The computer-implemented method of claim 1 , wherein:
the contribution estimate for the component computation is determined using a component computation approximation.
5 . The computer-implemented method of claim 4 , wherein:
the component computation approximation provides an estimate of an effect of the component computation on the composite computation.
6 . The computer-implemented method of claim 1 , further comprising:
generating a set of contribution estimates for a set of component computations, wherein the set of component computations include the component computation and are a part of the composite computation;
wherein: the set of component computations use a set of filters in a layer of a neural network; and the component computations in the set of component computations are selectively suppressed based on the set of contribution estimates and during the execution of the partial execution of the composite computation.
7 . The computer-implemented method of claim 1 , wherein:
the contribution estimate for the component computation is determined using a component computation approximation;
the component computation approximation is generated during a first execution of the composite computation and stored in a memory;
the partial execution of the composite computation is a second execution of the composite computation; and
the second execution is subsequent to the first execution.
8 . The computer-implemented method of claim 1 , wherein:
the component computation uses a filter from a layer of a neural network;
the neural network is a convolutional neural network; and
the operand is an input to the layer of the neural network.
9 . A computer-implemented method, comprising:
generating a degree of relevance for a component computation using an operand for the component computation, wherein the component computation is part of a composite computation; and
suppressing, based on the degree of relevance and during execution of a partial execution of the composite computation, the component computation;
wherein the suppressing is performed by a suppression logic block and comprises dropping the component computation from the composite computation by selectively providing operands from a set of operands to a math unit.
10 . The computer-implemented method of claim 9 , further comprising:
determining, based on the degree of relevance, that suppressing the component computation results in a decrease in computational resources consumed attributable to the partial execution of the composite computation that outweighs a decrease in accuracy of an output from the partial execution of the composite computation.
11 . The computer-implemented method of claim 9 , wherein:
the partial execution of the composite computation generates a first result for the composite computation and requires a first amount of computational resources;
a complete execution of the composite computation generates a second result for the composite computation and requires a second amount of computational resources;
the first result is not equal to the second result; and
the first amount is less than the second amount.
12 . The computer-implemented method of claim 9 , wherein:
the degree of relevance for the component computation is determined using a component computation approximation.
13 . The computer-implemented method of claim 12 , wherein:
the component computation approximation provides an estimate of an effect of the component computation on the composite computation.
14 . The computer-implemented method of claim 9 , further comprising:
generating a set of degrees of relevance for a set of component computations, wherein the set of component computations includes the component computation and are a part of the composite computation;
wherein the set of component computations use a set of filters in a layer of a neural network; and
wherein the component computations in the set of component computations are selectively suppressed based on the set of degrees of relevance and during the execution of the partial execution of the composite computation.
15 . The computer-implemented method of claim 9 , wherein:
the degree of relevance for the component computation is determined using a component computation approximation;
the component computation approximation is generated during a first execution of the composite computation and stored in a memory;
the partial execution of the composite computation is a second execution of the composite computation; and
the second execution is subsequent to the first execution.
16 . The computer-implemented method of claim 9 , wherein:
the component computation uses a filter from a layer of a neural network;
the neural network is a convolutional neural network; and
the operand is an input to the layer of the neural network.
17 . A processing core comprising:
a register file providing a set of operands for a composite computation, wherein the composite computation includes a component computation;
a set of math units; and
a suppression logic block programmed to:
generate a degree of relevance for the component computation using an operand for the component computation from the set of operands; and
suppress, based on the degree of relevance and during a partial execution of the composite computation, the component computation;
wherein the suppression logic block is programmed to suppress the component computation by selectively providing operands from the set of operands from the register file to the set of math units.
18 . The processing core of claim 17 , wherein:
the component computation uses a filter from a layer of a neural network;
the neural network is a convolutional neural network; and
the operand is an input to the layer of the neural network.
19 . The processing core of claim 17 , wherein:
the component computation uses a filter from a layer of a neural network;
the neural network is a convolutional neural network; and
the operand is a filter value of the filter.
20 . The computer-implemented method of claim 1 , wherein:
the composite computation is a sum-of-products computation; and
the component computation is a multiplication.
21 . The computer-implemented method of claim 1 , wherein:
generating the contribution estimate for the component computation using the operand comprises detecting that the operand is zero.
22 . The computer-implemented method of claim 1 , further comprising:
providing other operands from the set of operands to the math unit to complete the partial execution of the composite computation.
23 . The computer-implemented method of claim 1 , wherein:
the suppression logic block selectively provides the operands from the set of operands to the math unit from a register file.
24 . The computer-implemented method of claim 1 , wherein:
the set of operands are from one input tensor in a pair of input tensors that are used in the composite computation; and
the method further comprises generating a contribution estimate for every component computation in the composite computation.
25 . The computer-implemented method of claim 1 , wherein:
the math unit is a multiplier;
the operand is a multiplicand; and
the suppression logic block prevents less meaningful computations from being passed to the multiplier.
26 . The computer-implemented method of claim 1 , wherein:
the suppression logic block operates over multiple clock cycles to assure that all component computations in the composite computation that rise to a degree of relevance are executed.
27 . The computer-implemented method of claim 9 , wherein:
the composite computation is a sum-of-products computation; and
the component computation is a multiplication.
28 . The computer-implemented method of claim 9 , wherein:
generating the degree of relevance for the component computation using the operand comprises detecting that the operand is zero.
29 . The computer-implemented method of claim 9 , further comprising:
providing other operands from the set of operands to the math unit to complete the partial execution of the composite computation.
30 . The computer-implemented method of claim 9 , wherein:
the suppression logic block selectively provides the operands from the set of operands to the math unit from a register file.
31 . The computer-implemented method of claim 9 , wherein:
the set of operands are from one input tensor in a pair of input tensors that are used in the composite computation; and
the method further comprises generating a degree of relevance for every component computation in the composite computation.
32 . The computer-implemented method of claim 9 , wherein:
the math unit is a multiplier;
the operand is a multiplicand; and
the suppression logic block prevents less meaningful computations from being passed to the multiplier.
33 . The computer-implemented method of claim 9 , wherein:
the suppression logic block operates over multiple clock cycles to assure that all component computations in the composite computation that rise to a second degree of relevance are executed.
34 . The processing core of claim 17 , wherein:
the composite computation is a sum-of-products computation; and
the component computation is a multiplication.
35 . The processing core of claim 17 , wherein:
generating the degree of relevance for the component computation using the operand comprises detecting that the operand is zero.
36 . The processing core of claim 17 , wherein the suppression logic block is further programmed to:
provide other operands from the set of operands to the set of math units to complete the partial execution of the composite computation.
37 . The processing core of claim 17 , wherein:
the suppression logic block selectively provides the operands from the set of operands to the set of math units from the register file.
38 . The processing core of claim 17 , wherein:
the set of operands are from one input tensor in a pair of input tensors that are used in the composite computation; and
the suppression logic block is further programmed to generate a degree of relevance for every component computation in the composite computation.
39 . The processing core of claim 17 , wherein:
the suppression logic block is programmed to selectively provide the operands from the set of operands to the set of math units;
the set of math units includes at least one multiplier;
the operand is a multiplicand; and
the suppression logic block is programmed to prevent less meaningful computations from being passed to the at least one multiplier.
40 . The processing core of claim 17 , wherein:
the suppression logic block is programmed to selectively provide the operands from the set of operands to the set of math units; and
the suppression logic block is further programmed to operate over multiple clock cycles to assure that all component computations in the composite computation that rise to a second degree of relevance are executed.