Data processing method and apparatus, and related product for increased efficiency of tensor processing
View Patent ↗The present disclosure provides a data processing method and an apparatus and a related product. The products include a control module including an instruction caching unit, an instruction processing unit, and a storage queue unit. The instruction caching unit is configured to store computation instructions associated with an artificial neural network operation; the instruction processing unit is configured to parse the computation instructions to obtain a plurality of operation instructions; and the storage queue unit is configured to store an instruction queue, where the instruction queue includes a plurality of operation instructions or computation instructions to be executed in the sequence of the queue. By adopting the above-mentioned method, the present disclosure can improve the operation efficiency of related products when performing operations of a neural network model.
1. A data processing apparatus, comprising a controller and a processing circuit, wherein the controller is configured to:
determine that an operand of a first processing instruction includes an identifier of a descriptor, wherein the descriptor indicates a shape of tensor data on which the first processing instruction is to be executed;
obtain content of the descriptor from a descriptor storage space according to the identifier of the descriptor, wherein the content of the descriptor includes a shape vector comprising N shape parameters to indicate that the tensor data has N dimensions and each shape parameter indicates a feature of the tensor data in a corresponding dimension, where N is a positive integer; and
send the content of the descriptor and the first processing instruction to the processing circuit,
wherein the processing circuit is configured to obtain the tensor data based on a storage size of the tensor data determined based on the shape vector in the content of the descriptor and execute the first processing instruction on the tensor data.
2. The data processing apparatus of claim 1 , wherein to execute the first processing instruction on the tensor data obtained according to the content of the descriptor, the processing circuit is further configured to:
determine a data address of the tensor data in a data storage space corresponding to the operand of the first processing instruction according to the content of the descriptor;
obtain the tensor data from the data storage space based on the data address and the storage size of the tensor data; and
execute data processing on the tensor data corresponding to the first processing.
3. The data processing apparatus of claim 2 , wherein the operand further includes a data description position for the descriptor, wherein the processing circuit is configured to determine the data address of the tensor data corresponding to the operand in the data storage space according to the content of the descriptor and the data description position.
4. The data processing apparatus of claim 2 , wherein the content of the descriptor further includes at least one address parameter indicating the data address of the tensor data.
5. The data processing apparatus of claim 4 , wherein the address parameter includes a base address of a datum point of the descriptor in the data storage space of the tensor data, wherein the shape parameter includes at least one of:
a size of the data storage space in at least one of the N dimensions, a size of a storage area of the tensor data in at least one of the N dimensions, an offset of the storage area in at least one of the N dimensions, or a position of at least two vertices at diagonal positions in the N dimensions relative to the datum point.
6. The data processing apparatus of claim 3 , wherein the content of the descriptor further includes a mapping relationship between the data description position of the tensor data and the data address of the tensor data.
7. The data processing apparatus of claim 5 , wherein the base address includes a start address of the data storage space.
8. The data processing apparatus of claim 1 , wherein the first processing instruction includes a data access instruction, and the operand includes source data and target data,
wherein the controller is configured to:
obtain, when the source data includes an identifier of a first descriptor and the target data includes an identifier of a second descriptor, content of the first descriptor and content of the second descriptor from the descriptor storage space respectively; and
send the content of the first descriptor, the content of the second descriptor, and the first processing instruction to the processing circuit,
wherein the processing circuit is configured to:
obtain a first data address of the source data and a second data address of the target data respectively according to the content of the first descriptor and the content of the second descriptor; and
read data from the first data address and write the data to the second data address.
9. The data processing apparatus of claim 8 ,
wherein the controller is configured to:
obtain, when the source data includes an identifier of a third descriptor, content of the third descriptor from the descriptor storage space; and
send the content of the third descriptor and the first processing instruction to the processing circuit,
wherein the processing circuit is configured to:
obtain a third data address of the source data according to the content of the third descriptor; and
read data from the third data address and write the data to the second data address of the target data.
10. The data processing apparatus of claim 9 ,
wherein the controller is configured to:
obtain, when the target data includes an identifier of a fourth descriptor, content of a fourth descriptor from the descriptor storage space; and
send the content of the fourth descriptor and the first processing instruction to the processing circuit,
where the processing circuit is configured to:
obtain a fourth data address of the target data according to the content of the fourth descriptor, and
read data from the first data address of the source data and write the data to the fourth data address.
11. The data processing apparatus of claim 2 , wherein the first processing instruction includes an operation instruction, wherein
the processing circuit is configured to read data from the data address and execute an operation corresponding to the operation instruction.
12. The data processing apparatus of claim 1 , wherein the controller is further configured to:
decode the received first processing instruction to obtain a decoded first processing instruction, wherein the decoded first processing instruction includes an operation code and one or more operands, and the operation code is used to indicate a processing type corresponding to the first processing instruction.
13. The data processing apparatus of claim 2 , wherein the descriptor storage space is a storage space in an internal memory of the controller, and the data storage space is a storage space in an internal memory of the controller or a storage space in an external memory connected to the controller.
14. A data processing method performed by a processor, comprising:
determining, by a controller, that an operand of a first processing instruction includes an identifier of a descriptor, wherein the descriptor indicates a shape of tensor data on which the first processing instruction is to be executed;
obtaining, by the controller, content of the descriptor from a descriptor storage space according to the identifier of the descriptor, wherein the content of the descriptor includes a shape vector comprising N shape parameters to indicate that the tensor data has N dimensions and each shape parameter indicates a feature of the tensor data in a corresponding dimension, where N is a positive integer;
sending the content of the descriptor by the controller to a processing circuit;
determining, by the processing circuit, a data address of the tensor data corresponding to the operand of the first processing instruction in a data storage space according to the content of the descriptor;
determining, by the processing circuit, a storage size of the tensor data based on the shape vector in the content of the descriptor; and
according to the data address and the storage size of the tensor data, executing, by the processing circuit, data processing corresponding to the first processing instruction.
15. The data processing method of claim 14 , the operand further includes a data description position for the descriptor, wherein the determining a data address of the tensor data corresponding to the operand of the first processing instruction in the data storage space according to the content of the descriptor includes:
determining the data address of the tensor data corresponding to the operand in the data storage space according to the content of the descriptor and the data description position.
16. The data processing method of claim 14 , wherein the content of the descriptor further includes at least one address parameter indicating the data address of the tensor data, wherein the address parameter includes a base address of a datum point of the descriptor in the data storage space of the tensor data, wherein the shape parameter includes at least one of:
a size of the data storage space in at least one of the N dimensions, a size of a storage area of the tensor data in at least one of the N dimensions, an offset of the storage area in at least one of the N dimensions, or a position of at least two vertices at diagonal positions in the N dimensions relative to the datum point.
17. The data processing method of claim 15 , wherein the content of the descriptor further includes a mapping relationship between the data description position of the tensor data and the data address of the tensor data.
18. The data processing method of claim 14 , wherein the first processing instruction includes a data access instruction, and the operand includes source data and target data, wherein the method further comprises:
obtaining, when the source data includes an identifier of a first descriptor and the target data includes an identifier of a second descriptor, content of the first descriptor and content of the second descriptor from the descriptor storage space, respectively;
obtaining a first data address of the source data and a second data address of the target data according to the content of the first descriptor and the content of the second descriptor, respectively; and
reading data from the first data address and writing the data to the second data address.
19. A board card, comprising:
a storage device configured to store data;
a neural network chip;
an interface apparatus configured to implement data transmission between the neural network chip and an external equipment; and
a control device configured to monitor a state of the neural network chip,
wherein the neural network chip comprises a controller and a processing circuit, wherein the controller is configured to:
determine that an operand of a first processing instruction includes an identifier of a descriptor, wherein the descriptor indicates a shape of tensor data on which the first processing instruction is to be executed;
obtain content of the descriptor from a descriptor storage space according to the identifier of the descriptor, wherein the content of the descriptor includes a shape vector comprising N shape parameters to indicate that the tensor data has N dimensions and each shape parameter indicates a feature of the tensor data in a corresponding dimension, where N is a positive integer; and
send the content of the descriptor and the first processing instruction to the processing circuit,
wherein the processing circuit is configured to obtain the tensor data based on a storage size of the tensor data determined based on the shape vector in the content of the descriptor and execute the first processing instruction on the tensor data.
20. The board card of claim 19 , wherein
the storage device includes a plurality of groups of storage units, where each group of storage units is connected with the neural network chip by a bus, and the storage units are Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM),
the neural network chip includes a DDR controller configured to control data transmission and data storage of each storage unit, and
the interface apparatus is a standard Peripheral Component Interconnect Express (PCIE) interface.