Selective performance of deterministic computations for neural networks
View Patent ↗Selective performance of deterministic computations for neural networks is disclosed, including: obtaining a statistical model for a selection layer of the neural network, the statistical model indicating probabilities that corresponding values are selected by the selection layer, the statistical model being generated using historical data; selectively performing a subset of a plurality of deterministic computations on new input data to the neural network, the plurality of deterministic computations being associated with the deterministic computation layer, the selective performance of the deterministic computations being based at least in part on the statistical model and generating a computation result; and outputting the computation result to another layer in the neural network.
1. A system of reducing computation of a deterministic computation layer in a neural network, comprising:
a processor configured to:
obtain a statistical model for a selection layer of the neural network, the statistical model indicating probabilities that corresponding values are selected by the selection layer, the statistical model being generated using historical data;
selectively perform a subset of a plurality of deterministic computations on new input data to the neural network, the plurality of deterministic computations being associated with the deterministic computation layer, the selective performance of the deterministic computations being based at least in part on the statistical model and generating a computation result; and
output the computation result to another layer in the neural network; and
a memory coupled to the processor and configured to provide the processor with instructions.
2. The system of claim 1 , wherein to obtain the statistical model for the selection layer of the neural network comprises to:
use the neural network to analyze a set of data;
obtain a plurality of selections by the selection layer from the using of the neural network to analyze the set of data; and
generate the statistical model for the selection layer using the plurality of selections.
3. The system of claim 2 , wherein the set of data comprises one or more of the following: a set of training data, a set of validation data, a set of testing data, and a set of live data.
4. The system of claim 1 , wherein the deterministic computation layer precedes the selection layer in the neural network.
5. The system of claim 1 , wherein to selectively perform the subset of the plurality of the deterministic computations on the new input data to the neural network comprises to:
receive a plurality of masks corresponding to the plurality of deterministic computations, the plurality of masks being determined based at least in part on the statistical model; and
use the plurality of masks to determine which of the plurality of deterministic computations to perform on the new input data.
6. The system of claim 1 , wherein to selectively perform the subset of the plurality of the deterministic computations on the new input data to the neural network comprises to:
map a unit that is indicated by the statistical model as being not selected to a corresponding deterministic computation; and
omit performing the corresponding deterministic computation on the new input data.
7. The system of claim 1 , wherein the selection layer comprises a pooling layer.
8. The system of claim 1 , wherein the selection layer comprises a Rectified Linear Unit (ReLU) layer.
9. The system of claim 1 , wherein the other layer in the neural network to which the computation result is output is not the selection layer.
10. The system of claim 1 , wherein the plurality of deterministic computations comprises a plurality of convolution operations.
11. A method for reducing computation of a deterministic computation layer in a neural network, comprising:
obtaining a statistical model for a selection layer of the neural network, the statistical model indicating probabilities that corresponding values are selected by the selection layer, the statistical model being generated using historical data;
selectively performing a subset of a plurality of deterministic computations on new input data to the neural network, the plurality of deterministic computations being associated with the deterministic computation layer, the selective performance of the deterministic computations being based at least in part on the statistical model and generating a computation result; and
outputting the computation result to another layer in the neural network.
12. The method of claim 11 , wherein obtaining the statistical model for the selection layer of the neural network comprises:
using the neural network to analyze a set of data;
obtaining a plurality of selections by the selection layer from the using of the neural network to analyze the set of data; and
generating the statistical model for the selection layer using the plurality of selections.
13. The method of claim 12 , wherein the set of data comprises one or more of the following: a set of training data, a set of validation data, a set of testing data, and a set of live data.
14. The method of claim 11 , wherein the deterministic computation layer precedes the selection layer in the neural network.
15. The method of claim 11 , wherein selectively performing the subset of the plurality of the deterministic computations on the new input data to the neural network comprises:
receiving a plurality of masks corresponding to the plurality of deterministic computations, the plurality of masks being determined based at least in part on the statistical model; and
using the plurality of masks to determine which of the plurality of deterministic computations to perform on the new input data.
16. The method of claim 11 , wherein selectively performing the subset of the plurality of the deterministic computations on the new input data to the neural network comprises:
mapping a unit that is indicated by the statistical model as being not selected to a corresponding deterministic computation; and
omitting performing the corresponding deterministic computation on the new input data.
17. The method of claim 11 , wherein the selection layer comprises a pooling layer.
18. The method of claim 11 , wherein the selection layer comprises a Rectified Linear Unit (ReLU) layer.
19. A computer program product for reducing computation of a deterministic computation layer in a neural network, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
obtaining a statistical model for a selection layer of the neural network, the statistical model indicating probabilities that corresponding values are selected by the selection layer, the statistical model being generated using historical data;
selectively performing a subset of a plurality of deterministic computations on new input data to the neural network, the plurality of deterministic computations being associated with the deterministic computation layer, the selective performance of the deterministic computations being based at least in part on the statistical model and generating a computation result; and
outputting the computation result to another layer in the neural network.
20. The computer program product of claim 19 , wherein obtaining the statistical model for the selection layer of the neural network comprises:
using the neural network to analyze a set of data;
obtaining a plurality of selections by the selection layer from the using of the neural network to analyze the set of data; and
generating the statistical model for the selection layer using the plurality of selections.