Neural network processor, chip and electronic device
View Patent ↗The embodiments of the disclosure provide a neural network processor, a chip and an electronic device. The neural network processor includes a convolution processing unit, a vector processing unit, and an instruction issue module. The convolution processing unit and the vector processing unit are both connected to the instruction issue module. The instruction issue module is configured to issue a plurality of instructions to the convolution processing unit and the vector processing unit in parallel. The embodiments of the application can improve the efficiency of the neural network processor processing data.
1. A neural network processor, comprising:
a convolution processing unit;
a vector processing unit;
an instruction issue module, wherein the convolution processing unit and the vector processing unit are both in communication with the instruction issue module, and the instruction issue module is configured to issue a plurality of instructions to the convolution processing unit and the vector processing unit in parallel; and
a data storage module configured to store the data to-be-processed therein, wherein the data storage module is in communication with both of the convolution processing unit and the vector processing unit, and the convolution processing unit and the vector processing unit are configured to process the data to-be-processed to obtain one or more processing results and store the one or more processing results into the data storage module;
wherein the convolution processing unit and the vector processing unit are configured to process the data to-be-processed to obtain one or more processing results and store the one or more processing results into the data storage module, comprises:
the convolution processing unit performs a first processing on the data to-be-processed to obtain an intermediate processing result and directly transmits the intermediate processing result to the vector processing unit, the vector processing unit performs a second processing on the intermediate processing result to obtain the processing result and store the processing result into the data storage module; or
the convolution processing unit performs a first processing on the data to-be-processed to obtain an intermediate processing result and stores the intermediate processing result into the data storage module, the vector processing unit acquires the intermediate processing result from the data storage module, performs a second processing on the intermediate processing result to obtain the processing result, and stores the processing result into the data storage module.
2. The neural network processor according to claim 1 , wherein the instruction issue module is configured to issue the plurality of instructions to the convolution processing unit and the vector processing unit in parallel within one clock cycle.
3. The neural network processor according to claim 1 , wherein the instruction issue module is configured to issue the plurality of instructions to the convolution processing unit and the vector processing unit in parallel, according to types of the plurality of instructions.
4. The neural network processor according to claim 1 , further comprising a scalar processing unit in communication with the instruction issue module, wherein the instruction issue module is further configured to issue the plurality of instructions to the convolution processing unit and the scalar processing unit in parallel within one clock cycle.
5. The neural network processor according to claim 1 , further comprising a scalar processing unit in communication with the instruction issue module, wherein the instruction issue module is further configured to issue the plurality of instructions to the convolution processing unit and the scalar processing unit in parallel, according to types of the plurality of instructions.
6. The neural network processor according to claim 1 , further comprising a scalar processing unit in communication with the instruction issue module, wherein the instruction issue module is further configured to issue the plurality of instructions to the convolution processing unit, the vector processing unit and the scalar processing unit in parallel.
7. The neural network processor according to claim 6 , further comprising a shaping processing unit in communication with the instruction issue module, wherein the instruction issue module is configured to issue the plurality of instructions to the convolution processing unit, the vector processing unit, the scalar processing unit and the shaping processing unit in parallel.
8. The neural network processor according to claim 1 , wherein the plurality of instructions issued by the instruction issue module comprises a fine-grained instruction, the instruction issue module is configured to issue the fine-grained instruction to the convolution processing unit, and the convolution processing unit is configured to perform, according to the fine-grained instruction, a dot product operation on data received on the convolution processing unit.
9. The neural network processor according to claim 1 , wherein the data storage module is further in communication with the instruction issue module; the instruction issue module is further configured to:
issue the plurality of instructions to the convolution processing unit and the vector processing unit in parallel, in response to that the data storage module has data to-be-processed stored therein;
do not issue any instruction to the convolution processing unit and the vector processing unit, in response to that the data storage module does not store the data to-be-processed therein.
10. The neural network processor according to claim 1 , further comprising a scalar processing unit in communication with the instruction issue module, wherein the data storage module is further in communication with both of the scalar processing unit and the instruction issue module; the instruction issue module is further configured to:
issue the plurality of instructions to the convolution processing unit, the vector processing unit and the scalar processing unit in parallel, in response to that the data storage module has data to-be-processed stored therein;
do not issue any instruction to the convolution processing unit, the vector processing unit and the scalar processing unit, in response to that the data storage module does not store the data to-be-processed therein.
11. The neural network processor according to claim 1 , further comprising an instruction storage module in communication with the instruction issue module, wherein the instruction storage module is configured to store part or all instructions required to be issued by the instruction issue module.
12. The neural network processor according to claim 11 , further comprising:
a system bus interface, configured to be in communication with a system bus;
a data migration module, connected between the data storage module and the system bus interface, wherein the data migration module is configured to move data; and
an instruction migration module, connected between the instruction storage module and the system bus interface, wherein the instruction migration module is configured to move instructions.
13. The neural network processor according to claim 11 , further comprising:
a system bus interface, configured to be in communication with a system bus, the system bus interface being in communication with the instruction storage module; and
a data migration module, connected between the data storage module and the system bus interface, the data migration module being configured to move data.
14. The neural network processor according to claim 11 , further comprising:
a system bus interface, configured to be in communication with a system bus;
an instruction storage module, configured to store one or more instructions, the instruction storage module being in communication with the instruction issue module; and
a direct memory access, comprising at least one physical channel, at least one logical channel, and a first arbitration unit, wherein the at least one physical channel and at least one logical channel are in communication with the system bus interface through the first arbitration unit, the at least one physical channel is connected with the instruction storage module, and the at least one logical channel is connected with the data storage module.
15. A chip, comprising:
a system bus;
an external memory;
a central processing unit;
a neural network processor, connected to both of the external memory and the central processing unit via the system bus, wherein the neural network processor is controlled by the central processing unit to acquire data to-be-processed from the external memory, process the data to-be-processed to form a processing result, and feed back the processing result to the external memory;
wherein the neural network processor comprises:
a convolution processing unit;
a vector processing unit;
an instruction issue module, being in communication with both of the convolution processing unit and the vector processing unit, the instruction issue module being configured to issue a plurality of instructions to the convolution processing unit and the vector processing unit in parallel; and
a data storage module configured to store the data to-be-processed therein, wherein the data storage module is in communication with both of the convolution processing unit and the vector processing unit, and the convolution processing unit and the vector processing unit are configured to process the data to-be-processed to obtain one or more processing results and store the one or more processing results into the data storage module;
wherein the convolution processing unit and the vector processing unit are configured to process the data to-be-processed to obtain one or more processing results and store the one or more processing results into the data storage module, comprises:
the convolution processing unit performs a first processing on the data to-be-processed to obtain an intermediate processing result and directly transmits the intermediate processing result to the vector processing unit, the vector processing unit performs a second processing on the intermediate processing result to obtain the processing result and store the processing result into the data storage module; or
the convolution processing unit performs a first processing on the data to-be-processed to obtain an intermediate processing result and stores the intermediate processing result into the data storage module, the vector processing unit acquires the intermediate processing result from the data storage module, performs a second processing on the intermediate processing result to obtain the processing result, and stores the processing result into the data storage module.
16. An electronic device, comprising:
a system bus;
an external memory;
a central processing unit; and
a neural network processor, connected to both of the external memory and the central processing unit via the system bus, wherein the neural network processor is controlled by the central processing unit to acquire data to-be-processed from the external memory, process the data to-be-processed to form a processing result, and feed back the processing result to the external memory;
wherein the neural network processor comprises:
a convolution processing unit;
a vector processing unit;
an instruction issue module, being in communication with both of the convolution processing unit and the vector processing unit, the instruction issue module being configured to issue a plurality of instructions to the convolution processing unit and the vector processing unit in parallel; and
a data storage module configured to store the data to-be-processed therein, wherein the data storage module is in communication with both of the convolution processing unit and the vector processing unit, and the convolution processing unit and the vector processing unit are configured to process the data to-be-processed to obtain one or more processing results and store the one or more processing results into the data storage module;
wherein the convolution processing unit and the vector processing unit are configured to process the data to-be-processed to obtain one or more processing results and store the one or more processing results into the data storage module, comprises:
the convolution processing unit performs a first processing on the data to-be-processed to obtain an intermediate processing result and directly transmits the intermediate processing result to the vector processing unit, the vector processing unit performs a second processing on the intermediate processing result to obtain the processing result and store the processing result into the data storage module; or
the convolution processing unit performs a first processing on the data to-be-processed to obtain an intermediate processing result and stores the intermediate processing result into the data storage module, the vector processing unit acquires the intermediate processing result from the data storage module, performs a second processing on the intermediate processing result to obtain the processing result, and stores the processing result into the data storage module.