Method for optimizing computing power of neural network module, chip, electronic device and medium

1 . A method for optimizing computing power of a neural network and reducing real-time running latency of the neural network for image processing and image recognition on a chip, wherein the chip comprises a controller and a memory configured to store image data, wherein the method comprises:

obtaining, by the chip, a computational graph of the neural network having a plurality of operators, wherein each operator among the plurality of operators corresponds to a specific tensor operation executable by the chip;

inputting tensors carrying numerical values of image pixels into the computational graph of the neural network, wherein the numerical values of image pixels are stored in the memory of the chip;

determining, by the chip, a first operator among the plurality of operators, wherein the first operator corresponds to a tensor operation that changes a layout order of numerical values contained in an input tensor but does not change the numerical values themselves in the input tensor;

performing, by the chip based on the specific tensor operation of each operator, at least one adjustment on the first operator, wherein the at least one adjustment on the first operator is performed for merging or cancelling out with other operators in the computational graph to optimize the computational graph, and wherein performing the at least one adjustment on the first operator comprises:

counterchanging a position of the first operator with a subsequent operator or a preceding operator in the computational graph,

splitting the first operator into more than two identical operators, and

inserting operators on the computational graph, wherein the inserted operators correspond to mutually inverse tensor operations, and each of the inserted operators changes a layout order of numerical values contained in a corresponding tensor but does not change the numerical values themselves in the corresponding tensor;

determining, by the chip based on the specific tensor operation of each operator, a second operator in the computational graph, wherein the second operator is adjacent to the adjusted first operator or the inserted operators, wherein an operation of the adjusted first operator or one of the inserted operators and an operation of the second operator are capable of being merged or canceled out without affecting an operation result;

implementing a merge on the adjusted first operator or the one of the inserted operators and the second operator through a hardware read operation of the chip, wherein the hardware read operation of the chip is configured by the controller of the chip to replace execution of the adjusted first operator or the one of the inserted operators; and

generating an output result of the image recognition by executing tensor operations on the image data after implementing the merge of operators in the computational graph to optimize the computing power of the neural network and reduce the real-time running latency of the neural network.

2 . The method according to claim 1 , wherein the performing, by the chip, the at least one adjustment on a first operator in the computational graph comprises:

in response to the first operator having two or more subsequent operators and preceding operators that are branched, splitting the first operator into more than two identical first operators so that each first operator is in one-to-one correspondence with a subsequent operator or a preceding operator; and

in response to subsequent operators of two or more identical first operators being a same operator, unifying the two or more identical first operators into one operator.

3 . The method according to claim 1 , wherein the performing, by the chip, the merge on the adjusted first operator and the second operator, and executing operations of respective operators in the computational graph after the merge, comprises:

merging the adjusted first operator and the second operator into a fourth operator after the merge, in response to the operation of the second operator before the merge being a second operator operation performed on numerical values in an output tensor of the adjusted first operator,

wherein an operation of the fourth operator comprises:

reading, by the chip, the numerical values in the input tensor of the adjusted first operator in a predetermined order, to replace a tensor operation of the adjusted first operator with a reading operation, wherein the predetermined order is determined according to the tensor operation of the adjusted first operator; and

performing the second operator operation on the numerical values read by the chip according to the operation of the second operator.

4 . The method according to claim 1 , wherein the performing, by the chip, the merge on the adjusted first operator and the second operator, and executing operations of respective operators in the computational graph after the merge, comprises:

in response to the second operator being a tensor operation without changing numerical values in an input tensor before the merge or the cancellation, merging the adjusted first operator and the second operator into a third operator after the merge or the cancellation, wherein a tensor operation of the third operator implements both a tensor operation of the adjusted first operator and a tensor operation of the second operator.

5 . The method according to claim 1 , wherein the performing, by the chip, the merge on the adjusted first operator and the second operator, and executing operations of respective operators in the computational graph after the merge, comprises:

in response to the adjusted first operator and the second operator before the merge being mutually inverse operations, canceling out the adjusted first operator and the second operator.

6 . The method according to claim 1 , wherein the first operator is at least one selected from the group consisting of: transpose operator, reshape operator, broadcast operator, gather operator, reverse operator, concat operator, and cast operator.

7 . The method according to claim 1 , wherein the second operator is at least one selected from the group consisting of: transpose operator, reshape operator, broadcast operator, gather operator, reverse operator, concat operator, flatten operator, cast operator, elementwise binary operator, batch fully connected operator and convolution operator.

8 . An artificial intelligence chip, comprising:

a memory, configured to store received application data to be input into a computational graph of a neural network having a plurality of operators wherein each operator among the plurality of operators corresponds to a specific tensor operation executable by the artificial intelligence chip;

a controller, configured for:

inputting tensors carrying numerical values of image pixels into the computational graph of the neural network, wherein the numerical values of image pixels are stored in the memory of the artificial intelligence chip;

determining, by the artificial intelligence chip, a first operator among the plurality of operators, wherein the first operator corresponds to a tensor operation that changes a layout order of numerical values contained in an input tensor but does not change the numerical values themselves in the input tensor;

performing, by the artificial intelligence chip based on the specific tensor operation of each operator, at least one adjustment on the first operator, wherein the at least one adjustment on the first operator is performed for merging or cancelling out with other operators in the computational graph to optimize the computational graph, and wherein performing the at least one adjustment on the first operator comprises:

counterchanging a position of the first operator with a subsequent operator or a preceding operator in the computational graph,

splitting the first operator into more than two identical operators, and

inserting operators on the computational graph, wherein the inserted operators correspond to mutually inverse tensor operations, and each of the inserted operators changes a layout order of numerical values contained in a corresponding tensor but does not change the numerical values themselves in the corresponding tensor;

determining, by the by the artificial intelligence chip based on the specific tensor operation of each operator, a second operator in the computational graph, wherein the second operator is adjacent to the adjusted first operator or the inserted operators, wherein an operation of the adjusted first operator or one of the inserted operators and an operation of the second operator are capable of being merged or canceled out without affecting an operation result;

implementing a merge on the adjusted first operator or the one of the inserted operators and the second operator through a hardware read operation of the by the artificial intelligence chip, wherein the hardware read operation of the by the artificial intelligence chip is configured by the controller of the artificial intelligence chip to replace execution of the adjusted first operator or the one of the inserted operators; and

generating an output result of the image recognition by executing tensor operations on the image data after implementing the merge of operators in the computational graph to optimize the computing power of the neural network and reduce the real-time running latency of the neural network.

9 . The artificial intelligence chip according to claim 8 , wherein the performing, by the artificial intelligence chip, the at least one adjustment on a first operator in the computational graph comprises:

in response to the first operator having two or more subsequent operators and preceding operators that are branched, splitting the first operator into more than two identical first operators so that each first operator is in one-to-one correspondence with a subsequent operator or a preceding operator; and

in response to subsequent operators of two or more identical first operators being a same operator, unifying the two or more identical first operators into one operator.

10 . The artificial intelligence chip according to claim 8 , wherein the performing, by the artificial intelligence chip, the merge on the adjusted first operator and the second operator, and executing operations of respective operators in the computational graph after the merge, comprises:

merging the adjusted first operator and the second operator into a fourth operator after the merge, in response to the operation of the second operator before the merge being a second operator operation performed on numerical values in an output tensor of the adjusted first operator,

wherein an operation of the fourth operator comprises:

reading, by the artificial intelligence chip, the numerical values in the input tensor of the adjusted first operator in a predetermined order, to replace a tensor operation of the adjusted first operator with a reading operation, wherein the predetermined order is determined according to the tensor operation of the adjusted first operator; and

performing the second operator operation on the numerical values read by the artificial intelligence chip according to the operation of the second operator.

11 . The artificial intelligence chip according to claim 8 , wherein the performing, by the artificial intelligence chip, the merge on the adjusted first operator and the second operator, and executing operations of respective operators in the computational graph after the merge, comprises:

in response to the second operator being a tensor operation without changing numerical values in an input tensor before the merge, merging the adjusted first operator and the second operator into a third operator after the merge, wherein a tensor operation of the third operator implements both a tensor operation of the adjusted first operator and a tensor operation of the second operator.

12 . The artificial intelligence chip according to claim 8 , wherein the performing, by the artificial intelligence chip, the merge on the adjusted first operator and the second operator, and executing operations of respective operators in the computational graph after the merge, comprises:

in response to the adjusted first operator and the second operator before the merge being mutually inverse operations, canceling out the adjusted first operator and the second operator.

13 . The artificial intelligence chip according to claim 8 , wherein the first operator is at least one selected from the group consisting of: transpose operator, reshape operator, broadcast operator, gather operator, reverse operator, concat operator, and cast operator.

14 . The artificial intelligence chip according to claim 8 , wherein the second operator is at least one selected from the group consisting of: transpose operator, reshape operator, broadcast operator, gather operator, reverse operator, concat operator, flatten operator, cast operator, elementwise binary operator, batch fully connected operator and convolution operator.

15 . An electronic device for optimizing computing power of a neural network through a chip and reducing real-time running latency of the neural network for processing data on the chip, comprising:

a memory, configured to store instructions;

a processor, configured to read instructions in the memory and execute a method,

wherein the method comprises:

obtaining, by the chip, a computational graph of the neural network having a plurality of operators, wherein each operator among the plurality of operators corresponds to a specific tensor operation executable by the chip;

inputting tensors carrying numerical values of image pixels into the computational graph of the neural network, wherein the numerical values of image pixels are stored in the memory of the chip;

determining, by the chip, a first operator among the plurality of operators, wherein the first operator corresponds to a tensor operation that changes a layout order of numerical values contained in an input tensor but does not change the numerical values themselves in the input tensor;

performing, by the chip based on the specific tensor operation of each operator, at least one adjustment on the first operator, wherein the at least one adjustment on the first operator is performed for merging or cancelling out with other operators in the computational graph to optimize the computational graph, and wherein performing the at least one adjustment on the first operator comprises:

counterchanging a position of the first operator with a subsequent operator or a preceding operator in the computational graph,

splitting the first operator into more than two identical operators, and

inserting operators on the computational graph, wherein the inserted operators correspond to mutually inverse tensor operations, and each of the inserted operators changes a layout order of numerical values contained in a corresponding tensor but does not change the numerical values themselves in the corresponding tensor;

determining, by the chip based on the specific tensor operation of each operator, a second operator in the computational graph, wherein the second operator is adjacent to the adjusted first operator or the inserted operators, wherein an operation of the adjusted first operator or one of the inserted operators and an operation of the second operator are capable of being merged or canceled out without affecting an operation result;

implementing a merge on the adjusted first operator or the one of the inserted operators and the second operator through a hardware read operation of the chip, wherein the hardware read operation of the chip is configured by the controller of the chip to replace execution of the adjusted first operator or the one of the inserted operators; and

generating an output result of the image recognition by executing tensor operations on the image data after implementing the merge of operators in the computational graph to optimize the computing power of the neural network and reduce the real-time running latency of the neural network.

16 . A non-transitory computer-readable storage medium, having instructions stored thereon;

wherein, the instructions, when read by a processor, cause the processor to execute the method according to claim 1 .