Neural network optimization method, electronic device and processor
The present invention discloses a neural network optimization method. An operator to be replaced is selected from multiple operators in a network layer according to a predetermined condition, and the operator to be replaced is replaced by multiple equivalent operators according to a calculation function corresponding to the operator to be replaced, wherein the multiple equivalent operators include a target operator. Pre-calculating is performed for a first operator among the multiple equivalent operators, and the calculation result is inputted into the target operator. A second operator is identified according to data change conditions of the multiple equivalent operators, and the second operator is combined with the target operator to complete optimization of a neural network model. The present invention can further perform lossless conversion of the operators in the neural network, further improving calculation performance on the basis of a simplified network structure.
1. A neural network optimization method for optimizing a neural network for operating on a device having a hardware constrained calculation platform performed by a processor, comprising:
selecting an operator to be replaced from a plurality of operators in a network layer according to a predetermined condition;
replacing the operator to be replaced by a plurality of equivalent operators according to a calculation function corresponding to the operator to be replaced, wherein the plurality of equivalent operators comprises a target operator;
pre-calculating for a first operator among the plurality of equivalent operators to complete a calculation function of the first operator, and inputting a calculation result into the target operator;
identifying a second operator according to data change conditions of the plurality of equivalent operators, and combining the second operator with the target operator; and
deleting the first operator,
wherein a first power needed by the processor to operate the neural network prior to applying the optimization method is greater than a second power needed by the processor to operate the neural network after applying the optimization method, and
wherein a first granularity of the neural network prior to applying the optimization method and a second granularity of the neural network after applying the optimization method are the same.
2. The neural network optimization method according to claim 1 , wherein the first operator is a constant operator.
3. The neural network optimization method according to claim 1 , wherein an amount of data of the second operator before and after operation does not change.
4. The neural network optimization method according to claim 1 , wherein a neural network having been optimized by the neural network optimization method is applied to a calculation platform, and the predetermined condition comprises a function that is not supported by the calculation platform.
5. The neural network optimization method according to claim 1 , wherein the step of replacing the operator to be replaced by the plurality of equivalent operators according to the calculation function corresponding to the operator to be replaced comprises:
calculating feature information of the target operator according to a weight and a bias of the operator to be replaced; and
replacing the operator to be replaced by the plurality of equivalent operators according to the calculation function and the feature information.
6. A processor, for executing a program code to implement a neural network optimization method for optimizing a neural network for operating on a device having a hardware constrained calculation platform, the neural network optimization method comprising:
selecting an operator to be replaced from a plurality of operators in a network layer according to a predetermined condition;
replacing the operator to be replaced by a plurality of equivalent operators according to a calculation function corresponding to the operator to be replaced, wherein the plurality of equivalent operators comprises a target operator;
pre-calculating for a first operator among the plurality of equivalent operators to complete a calculation function of the first operator, and inputting a calculation result into the target operator;
identifying a second operator according to data change conditions of the plurality of equivalent operators, and combining the second operator with the target operator; and
deleting the first operator,
wherein a first power needed by the processor to operate the neural network prior to applying the optimization method is greater than a second power needed by the processor to operate the neural network after applying the optimization method, and
wherein a first granularity of the neural network prior to applying the optimization method and a second granularity of the neural network after applying the optimization method are the same.
7. The processor according to claim 6 , wherein the first operator is a constant operator.
8. The processor according to claim 6 , an amount of data of the second operator before and after operation does not change.
9. An electronic device, comprising a neural network calculation device, the neural network calculation device operating a neural network model having been optimized by a neural network optimization method, the neural network optimization method for optimizing a neural network for operating on the calculation device having a hardware constrained calculation platform comprising:
selecting an operator to be replaced from a plurality of operators in a network layer according to a predetermined condition;
replacing the operator to be replaced by a plurality of equivalent operators according to a calculation function corresponding to the operator to be replaced, wherein the plurality of equivalent operators comprises a target operator;
pre-calculating for a first operator among the plurality of equivalent operators to complete a calculation function of the first operator, and inputting a calculation result into the target operator;
identifying a second operator according to data change conditions of the plurality of equivalent operators, and combining the second operator with the target operator; and
deleting the first operator,
wherein a first power needed by the device to operate the neural network prior to applying the optimization method is greater than a second power needed by the device to operate the neural network after applying the optimization method, and
wherein a first granularity of the neural network prior to applying the optimization method and a second granularity of the neural network after applying the optimization method are the same.
10. The electronic device according to claim 9 , wherein the first operator is a constant operator.
11. The electronic device according to claim 9 , an amount of data of the second operator before and after operation does not change.