Accelerator for deep neural networks
A system for bit-serial computation in a neural network is described. The system may be embodied on an integrated circuit and include one or more bit-serial tiles for performing bit-serial computations in which each bit-serial tile receives input neurons and synapses, and communicates output neurons. Also included is an activation memory for storing the neurons and a dispatcher and a reducer. The dispatcher reads neurons and synapses from memory and communicates either the neurons or the synapses bit-serially to the one or more bit-serial tiles. The other of the neurons or the synapses are communicated bit-parallelly to the one or more bit-serial tiles, or according to a further embodiment, may also be communicated bit-serially to the one or more bit-serial tiles. The reducer receives the output neurons from the one or more tiles, and communicates the output neurons to the activation memory.
1. A system for bit-serial computation in a neural network, comprising:
one or more bit-serial tiles for performing bit-serial computations in a neural network, each bit-serial tile receiving input neurons and synapses, and generating output neurons;
an activation memory for storing neurons and in communication with the one or more bit-serial tiles via a dispatcher and a reducer,
wherein the dispatcher reads neurons from the activation memory and communicates the neurons to the one or more bit-serial tiles via a first interface,
and wherein the dispatcher reads synapses from a memory and communicates the synapses to the one or more bit-serial tiles via a second interface,
and wherein the reducer receives the output neurons from the one or more bit-serial tiles, and communicates the output neurons to the activation memory via a third interface;
and wherein one of the first interface and the second interface communicates the neurons or the synapses to the one or more bit-serial tiles bit-serially and the other of the first interface and the second interface communicates the neurons or the synapses to the one or more bit-serial tiles bit-parallelly;
and wherein the one or more bit-serial tiles is configured to process bit-serial inputs and bit-parallel inputs by independently processing a bit-parallel input with each bit of a bit-serial input to produce an initial product for use in producing a partial output neuron.
2. The system of claim 1 , wherein the dispatcher sends one or more offsets identifying the one-bits in a block of one or more neurons, thereby skipping the zero-bits of the block.
3. The system of claim 1 , wherein the dispatcher comprises a shuffler to collect the neurons in one or more bricks and a transposer to convert the bricks into serial bit streams.
4. The system of claim 1 , wherein the bit-serial tiles each further comprise a synapse buffer, an input neuron buffer holding input neurons from the dispatcher and a neuron output buffer holding output neurons pending communication to the reducer.
5. The system of claim 1 , wherein the reducer further converts the output neurons from a full precision value to a closest reduced precision value prior to storing.
6. The system of claim 1 , wherein the reducer further converts the output neurons into one or more bricks according to a desired precision.
7. The system of claim 1 , wherein the dispatcher reduces the precision of an input neuron, based on a most significant bit value or a least significant bit value of the input neuron.
8. The system of claim 7 , wherein the dispatcher reduces the precision of the input neuron based on the most significant bit value and the least significant bit value of the input neuron.
9. The system of claim 1 , wherein the processing time for the bit-serial computations by the one or more bit-serial tiles is proportional to the number of bits processed from the neurons.
10. The system of claim 1 , wherein the one or more bit-serial tiles process synapses bit-parallelly.
11. The system of claim 1 , wherein the one or more bit-serial tiles comprise a weight register.
12. The system of claim 11 , wherein the one or more bit-serial tiles further comprise a serial weight register.
13. The system of claim 12 , wherein the processing time for the bit-serial computations by the one or more bit-serial tiles is proportional to the product of the number of bits processed from the neurons and synapses.
14. The system of claim 1 , further comprising a controller for controlling the operation of the system.
15. The system of claim 1 , further comprising a first serial inner product subunit which selectively uses an output of a second serial inner product subunit as an input.
16. The system of claim 15 , wherein the first serial inner product subunit which selectively uses an output based on one or more dimensions of a layer in a neural network.
17. The system of claim 2 , wherein the dispatcher processes the one or more neurons in subgroups of bits to identify the one-bits in the neuron.
18. The system of claim 2 , wherein the one or more neurons are encoded using positive and negative powers of two prior to communicating the offsets of one-bits.
19. The system of claim 2 , wherein at least two or more of the one or more neurons are processed concurrently and are grouped into subgroups.
20. The system of claim 19 , wherein at least one of the subgroups can advance independently to process a new set of neuron values without waiting for other concurrently processed subgroups to advance.
21. The system of claim 2 , wherein at least one least significant bit identified in the one or more offsets identifying the one-bits in a block of one or more neurons is dropped and not processed by a serial inner product subunit.
22. The system of claim 2 , wherein the one or more bit-serial tiles encode the one or more neurons into one-bit offsets.
23. The system of claim 1 , wherein the neurons or synapses are stored in memory at a reduced precision.
24. The system of claim 23 , wherein the neurons or synapses are stored in blocks, each block having a starting memory address, wherein the starting memory address is specified in an array elsewhere in memory.
25. The system of claim 1 , further comprising one or more serial inner product subunits, wherein each serial inner product subunit processes a block of neurons or synapses and one or more offsets identifying the one-bits in the block of neurons or synapses, and skips at least one zero-bit in the block.
26. A system for bit-serial computation in a neural network, comprising:
one or more bit-serial tiles for performing bit-serial computations in a neural network, each bit-serial tile receiving input neurons and synapses, and communicating output neurons;
an activation memory for storing neurons and in communication with the one or more bit-serial tiles via a dispatcher and a reducer,
wherein the dispatcher reads neurons from the activation memory and communicates the neurons to the one or more bit-serial tiles via a first interface,
and wherein the dispatcher reads synapses from a memory and communicates the synapses to the one or more bit-serial tiles via a second interface,
and wherein the reducer receives the output neurons from the one or more bit-serial tiles, and communicates the output neurons to the activation memory via a third interface;
and wherein the first interface and the second interface communicate the neurons and the synapses to the one or more bit-serial tiles bit-serially;
and wherein the one or more bit-serial tiles is configured to process inputs by independently processing each bit of a neuron with each bit of a synapse to produce an initial product for use in producing a partial output neuron.
27. The system of claim 26 , wherein the dispatcher reduces the precision of an input synapse, based on a most significant bit value or a least significant bit value of the input neuron.
28. The system of claim 26 , wherein the dispatcher reduces the precision of the input synapse based on the most significant bit value and the least significant bit value of the input neuron.
29. An integrated circuit comprising a bit-serial neural network accelerator, the integrated circuit comprising:
one or more bit-serial tiles for performing bit-serial computations in a neural network,
each bit-serial tile receiving input neurons and synapses, and generating output neurons;
an activation memory for storing neurons and in communication with the one or more bit-serial tiles via a dispatcher and a reducer,
wherein the dispatcher reads neurons from the activation memory and communicates the neurons to the one or more bit-serial tiles via a first interface,
and wherein the dispatcher reads synapses from a memory and communicates the synapses to the one or more bit-serial tiles via a second interface,
and wherein the reducer receives the output neurons from the one or more bit-serial tiles, and communicates the output neurons to the activation memory via a third interface;
and wherein one of the first interface and the second interface communicates the neurons or the synapses to the one or more bit-serial tiles bit-serially and the other of the first interface and the second interface communicates the neurons or the synapses to the one or more bit-serial tiles bit-parallelly;
and wherein the one or more bit-serial tiles is configured to process bit-serial inputs and bit-parallel inputs by independently processing a bit-parallel input with each bit of a bit-serial input to produce an initial product for use in producing a partial output neuron.
30. An integrated circuit comprising a bit-serial neural network accelerator, the integrated circuit comprising:
one or more bit-serial tiles for performing bit-serial computations in a neural network, each bit-serial tile receiving input neurons and synapses, and communicating output neurons;
an activation memory for storing neurons and in communication with the one or more bit-serial tiles via a dispatcher and a reducer,
wherein the dispatcher reads neurons from the activation memory and communicates the neurons to the one or more bit-serial tiles via a first interface,
and wherein the dispatcher reads synapses from a memory and communicates the synapses to the one or more bit-serial tiles via a second interface,
and wherein the reducer receives the output neurons from the one or more bit-serial tiles, and communicates the output neurons to the activation memory via a third interface;
and wherein the first interface and the second interface communicate the neurons and the synapses to the one or more bit-serial tiles bit-serially;
and wherein the one or more bit-serial tiles is configured to process inputs by independently processing each bit of a neuron with each bit of a synapse to produce an initial product for use in producing a partial output neuron.