Special purpose neural network training chip
Methods, systems, and apparatus including a special purpose hardware chip for training neural networks are described. The special-purpose hardware chip may include a scalar processor configured to control computational operation of the special-purpose hardware chip. The chip may also include a vector processor configured to have a 2-dimensional array of vector processing units which all execute the same instruction in a single instruction, multiple-data manner and communicate with each other through load and store instructions of the vector processor. The chip may additionally include a matrix multiply unit that is coupled to the vector processor configured to multiply at least one two-dimensional matrix with a second one-dimensional vector or two-dimensional matrix in order to obtain a multiplication result.
1 . A special-purpose hardware chip for training neural networks, the special-purpose hardware chip comprising:
a scalar processor configured to control computational operation of the special-purpose hardware chip;
a vector processor having a 2-dimensional array of vector processing units; and
a matrix multiply unit that is coupled to the vector processor and configured to multiply at least a first two-dimensional matrix with a first one-dimensional vector or a second two-dimensional matrix in order to obtain a multiplication result,
wherein the vector processor includes a plurality of lanes, wherein each vector processing unit of the 2-dimensional array of vector processing units in the vector processor is located in a respective lane of the plurality of lanes, wherein one or more vector processing units of the 2-dimensional array of vector processing units that are located in the same lane are configured to communicate with one another through respective load and store instructions.
2 . The special-purpose hardware chip of claim 1 , further comprising:
a vector memory configured to provide private memory to the vector processor.
3 . The special-purpose hardware chip of claim 1 , further comprising:
a scalar memory configured to provide private memory to the scalar processor.
4 . The special-purpose hardware chip of claim 1 , further comprising:
a transpose unit configured to perform a transposition operation of a matrix.
5 . The special-purpose hardware chip of claim 1 , further comprising:
a reduction and permutation unit configured to perform a reduction on numbers and permute the numbers among different lanes of a vector array.
6 . The special-purpose hardware chip of claim 1 , further comprising:
a first memory configured to store data of the special-purpose hardware chip.
7 . The special-purpose hardware chip of claim 1 , further comprising a sparse computation core.
8 . The special-purpose hardware chip of claim 1 , further comprising:
an interface; and
an inter-chip interconnect, which connects the interface or resources on the special purpose hardware chip to other special-purpose hardware chips or resources.
9 . The special-purpose hardware chip of claim 8 , further comprising:
a first memory, wherein the inter-chip interconnect connects the interface and the first memory to other special-purpose hardware chips.
10 . The special-purpose hardware chip of claim 8 , wherein the interface is a host interface to a host computer.
11 . The special-purpose hardware chip of claim 8 , wherein the interface is a standard network interface to a network of host computers.
12 . The special-purpose hardware chip of claim 8 , further comprising a scalar memory and a vector memory.
13 . The special-purpose hardware chip of claim 8 , wherein a scalar instruction set of the instructions includes arithmetic operations used in address calculations, load/store instructions, and branch instructions, and wherein the remaining instructions of the instructions encode instructions for the vector processor and the matrix multiply unit.
14 . The special-purpose hardware chip of claim 1 , wherein each vector processing unit in the 2-dimensional array of vector processing units includes 32 registers.
15 . The special-purpose hardware chip of claim 1 , wherein each vector processing unit in the 2-dimensional array of vector processing units is configured to perform at least one of a floating point operation or integer operation.
16 . The special-purpose hardware chip of claim 1 , wherein each vector processing unit in the 2-dimensional array of vector processing units is configured to execute two respective arithmetic logic unit (ALU) instructions, a respective load instruction, and a respective store instruction in each clock cycle.
17 . The special-purpose hardware chip of claim 16 , wherein each vector processing unit in the 2-dimensional array of vector processing units is configured to compute respective offset memory addresses for executing the respective load and store instructions in each clock cycle.