Exploiting input data sparsity in neural network compute units

1. A computer-implemented method for performing neural network computations for a neural network comprising a plurality of layers, the method comprising:

receiving, by a computing device, multiple inputs for processing at a first layer of the neural network, the multiple inputs being provided, at least in part, from a source external to the computing device;

determining, by a controller of the computing device, whether each of the multiple inputs has one of a zero value or a non-zero value;

determining, by the controller, a mapping of a subset of inputs to address locations for a memory bank at the computing device based on the determined value of each input, wherein inputs having a zero value are excluded from the subset;

storing, in the memory bank, the subset of inputs using the determined mapping of the subset of inputs to address locations for the memory bank; and

providing, by the controller and from the memory bank, an input from the subset of inputs to a data bus coupled to one or more multiply accumulate units of a computational array, wherein the input is provided from a memory address location to which the subset of inputs are mapped.

2. The method of claim 1 , wherein determining the mapping of the subset of inputs comprises:

generating a bitmap comprising a plurality of bits, wherein each bit of the plurality of bits of the bitmap indicates at least one of a non-zero input value or a zero input value; and

wherein storing the subset of inputs comprises storing the subset of inputs based on the bitmap.

3. The method of claim 1 , further including:

providing the input based on the determined value of the input,

performing, by a first multiply accumulate unit, a computation using the determined value of the input,

subsequently providing a second input that has a zero value, and

preventing, in a second multiply accumulate unit, a computation that would otherwise be performed using the zero value.

4. The method of claim 3 , wherein preventing occurs in response to the controller determining that the second input is provided from a memory address location that does not store an input having a value that is greater than zero.

5. The method of claim 4 , further including:

detecting, by the controller, that the second input is provided from a memory address location that is not associated with a memory address location to which the subset of inputs are mapped, and

in response to detecting, providing a control signal to the second multiply accumulate unit to prevent a multiply operation using the second input.

6. The method of claim 1 , wherein the method further comprises:

mapping, by the controller and to a first multiply accumulate unit, a first portion of a tensor computation that uses a first input in the subset of inputs, and

mapping, to a second multiply accumulate unit that differs from the first multiply accumulate unit, a second portion of the tensor computation that also uses the first input in the subset of inputs.

7. The method of claim 1 , further comprising, sequentially providing a single input onto the data bus, the single input being obtained from memory address locations to which the subset of inputs are mapped.

8. The method of claim 1 , wherein providing the input from the subset of inputs to the data bus comprises, not providing inputs that have a zero value.

9. One or more non-transitory machine-readable storage devices for storing instructions that are executable by one or more processing devices to cause performance of operations comprising:

receiving, by a computing device, multiple inputs for processing at a first layer of a neural network, the multiple inputs being provided, at least in part, from a source external to the computing device;

determining, by a controller of the computing device, whether each of the multiple inputs has one of a zero value or a non-zero value;

determining, by the controller, a mapping of a subset of inputs to address locations for a memory bank at the computing device based on the determined value of each input, wherein inputs having a zero value are excluded from the subset;

storing, in the memory bank, the subset of inputs using the determined mapping of the subset of inputs to address locations for the memory bank; and

providing, by the controller and from the memory bank, an input from the subset of inputs to a data bus coupled to one or more multiply accumulate units of a computational array, wherein the input is provided from a memory address location to which the subset of inputs are mapped.

10. The machine-readable storage devices of claim 9 , wherein determining the mapping of the subset of inputs comprises:

generating a bitmap comprising a plurality of bits, wherein each bit of the plurality of bits of the bitmap indicates at least one of a non-zero input value or a zero input value; and

wherein storing the subset of inputs comprises storing the subset of inputs based on the bitmap.

11. The machine-readable storage devices of claim 9 , wherein the operations further comprise, providing the input based on the determined value of the input,

performing, by a first multiply accumulate unit, a computation using the determined value of the input,

subsequently providing a second input that has a zero value, and

preventing, in a second multiply accumulate unit, a computation that would otherwise be performed using the zero value.

12. The machine-readable storage devices of claim 11 , wherein preventing occurs in response to the controller determining that the second input is provided from a memory address location that does not store an input having a value that is greater than zero.

13. The machine-readable storage devices of claim 12 , wherein the operations further comprise:

detecting, by the controller, that the second input is provided from a memory address location that is not associated with a memory address location to which the subset of inputs are mapped, and

in response to detecting, providing a control signal to the second multiply accumulate unit to prevent a multiply operation using the second input.

14. The machine-readable storage devices of claim 9 , wherein the operations further comprise:

mapping, by the controller and to a first multiply accumulate unit, a first portion of a tensor computation that uses a first input in the subset of inputs, and

mapping, to a second multiply accumulate unit that differs from the first multiply accumulate unit, a second portion of the tensor computation that also uses the first input in the subset of inputs.

15. An electronic system for performing neural network computations for a neural network comprising a plurality of layers, the electronic system comprising:

a controller located in a computing device, the controller including one or more processing devices; and

one or more non-transitory machine-readable storage devices for storing instructions that are executable by the one or more processing devices to cause performance of operations comprising:

receiving, by a computing device, multiple inputs for processing at a first layer of the neural network, the multiple inputs being provided, at least in part, from a source external to the computing device;

determining, by the controller, whether each of the multiple inputs has one of a zero value or a non-zero value;

determining, by the controller, a mapping of a subset of inputs to address locations for a memory bank at the computing device based on the determined value of each input, wherein inputs having a zero value are excluded from the subset;

storing, in the memory bank, the subset of inputs using the determined mapping of the subset of inputs to address locations for the memory bank; and

providing, by the controller and from the memory bank, an input from the subset of inputs to a data bus coupled to one or more multiply accumulate units of a computational array, wherein the input is provided from a memory address location to which the subset of inputs are mapped.

16. The electronic system of claim 15 , wherein determining the mapping of the subset of inputs comprises:

generating a bitmap comprising a plurality of bits, wherein each bit of the plurality of bits of the bitmap indicates at least one of a non-zero input value or a zero input value; and

wherein storing the subset of inputs comprises storing the subset of inputs based on the bitmap.

17. The electronic system of claim 15 , wherein the operations further comprise, providing the input based on the determined value of the input,

performing, by a first multiply accumulate unit, a computation using the determined value of the input,

subsequently providing a second input that has a zero value, and

preventing, in a second multiply accumulate unit, a computation that would otherwise be performed using the zero value.

18. The electronic system of claim 17 , wherein preventing occurs in response to the controller determining that the second input is provided from a memory address location that does not store an input having a value that is greater than zero.

19. The electronic system of claim 17 , wherein the operations further comprise:

detecting, by the controller, that the second input is provided from a memory address location that is not associated with a memory address location to which the subset of inputs are mapped, and

in response to detecting, providing a control signal to the second multiply accumulate unit to prevent a multiply operation using the second input.

20. The electronic system of claim 15 , wherein the operations further comprise:

mapping, by the controller and to a first multiply accumulate unit, a first portion of a tensor computation that uses a first input in the subset of inputs, and

mapping, to a second multiply accumulate unit that differs from the first multiply accumulate unit, a second portion of the tensor computation that also uses the first input in the subset of inputs.