Inference apparatus, convolution operation execution method, and program
An inference apparatus comprises a plurality of PEs (Processing Elements) and a control part. The control part operates a convolution operation in a convolutional neural network using each of a plurality of pieces of input data and a weight group including a plurality of weights corresponding to each of the plurality of pieces of input data by controlling the plurality of PEs. Further, each of the plurality of PEs executes a computation including multiplication of a single piece of the input data by a single weight and also executes multiplication included in the convolution operation using an element with a non-zero value included in each of the plurality of pieces of input data.
1. An inference apparatus, comprising:
a plurality of PEs (Processing Elements); and
a control part that comprises:
an evaluation index calculation section that calculates an evaluation index for evaluating a plurality of weight groups that each include a plurality of weights that each correspond to a plurality of pieces of input data;
a weight group sorting section that sorts an order of the weight groups to be multiplied by the plurality of pieces of input data based on the evaluation index; and
an order information notification section that notifies layers of order information regarding sorting of the weight groups;
wherein the control part operates a convolution operation in a convolutional neural network using each of a the plurality of pieces of input data and each of the plurality of weight groups by controlling the plurality of PEs; and
wherein each of the plurality of PEs executes a computation including multiplication of a single piece of the input data by a single weight and multiplication included in the convolution operation using an element with a non-zero value included in each of the plurality of pieces of input data.
2. The inference apparatus according to claim 1 , wherein
the control part controls the plurality of PEs so as to perform in parallel some multiplications between each of the plurality of pieces of input data and each weight corresponding to each of the plurality of pieces of input data.
3. The inference apparatus according to claim 2 , wherein
the control part controls the plurality of PEs so as to execute in parallel the some multiplications between each of the plurality of pieces of input data and each of the plurality of weights, where the weights used in the some of multiplications belong to different weight groups but correspond to a same piece of input data.
4. The inference apparatus according to claim 3 , further comprising an input data division section that divides the plurality of pieces of input data and supplies the divided pieces of input data to the PEs that perform the some of multiplications in parallel.
5. The inference apparatus according to claim 3 , further comprising a standby section in which the computation stands by until the plurality of PEs that perform the some of multiplications in parallel have finished performing the some of multiplications.
6. The inference apparatus according to claim 3 , further comprising a result addition section that, for each weight group, adds together results of multiplications between each of the plurality of pieces of input data and the plurality of weights included in the each weight group.
7. The inference apparatus according to claim 1 , wherein
each of the plurality of PEs comprises:
an input data processing section that identifies a non-zero element having the non-zero value and the position of the non-zero element in received pieces of input data;
a weight readout section that reads a value at the position corresponding to the position of the non-zero element from the weights to be multiplied by the received pieces of input data from a storage medium that stores the plurality of weight groups; and
a multiplication/addition section that multiplies the non-zero element by the value read by the weight readout section;
wherein the multiplication/addition section adds together multiplication results upon completion.
8. The inference apparatus according to claim 7 , wherein
each of the plurality of PEs further comprises a computation result storage section that stores addition results computed by the multiplication/addition section.
9. The inference apparatus according to claim 3 , wherein
the control part causes each of the plurality of PEs to execute a plurality of processes of multiplying a received piece of input data by the weights included in different weight groups in time division.
10. The inference apparatus according to claim 1 , wherein
the control part further comprises:
an order information acquisition section that acquires the notified order information; and
a weight sorting section that changes associations between the pieces of input data and the plurality of weights included in the weight groups based on the order information.
11. A convolution operation execution method for an inference apparatus that includes a control part and a plurality of PEs (Processing Elements) each executing a computation including multiplication of a single piece of input data by a single weight and that operates a convolution operation in a convolutional neural network using each of a plurality of pieces of input data and a weight group including a plurality of weights that each correspond to each of the plurality of pieces of input data, the convolution operation execution method including:
causing the control part to calculate an evaluation index for evaluating weight groups including the weight group;
causing the control part to sort an order of the weight groups to be multiplied by the plurality of pieces of input data based on the evaluation index;
causing the control part to notify layers of order information regarding sorting of the weight groups;
causing each of the plurality of PEs to identify an element with a non-zero value included in each of the plurality of pieces of input data; and
causing each of the plurality of PEs to execute multiplication included in the convolution operation using the identified element.
12. A non-transitory computer-readable storage medium storing a program causing a computer to execute:
calculating an evaluation index for evaluating weight groups that each include a plurality of weights that each correspond to a plurality of pieces of input data;
sorting an order of the weight groups to be multiplied by the plurality of pieces of input data based on the evaluation index;
notifying layers of order information regarding sorting of the weight groups;
identifying an element with a non-zero value included in each of the plurality of pieces of input data; and
executing multiplication included in the convolution operation using the identified element,
wherein the computer being is provided in an inference apparatus that includes a plurality of PEs (Processing Elements) each executing a computation including multiplication of a single piece of input data by a single weight and that operates a convolution operation in a convolutional neural network using each of the plurality of pieces of input data and one of the weight groups.
13. The convolution operation execution method according to claim 11 , including:
controlling the plurality of PEs so as to perform in parallel some multiplications between each of the plurality of pieces of input data and each weight corresponding to each of the plurality of pieces of input data.
14. The convolution operation execution method according to claim 13 , including:
controlling the plurality of PEs so as to execute in parallel the some multiplications between each of the plurality of pieces of input data and each of the plurality of weights, where the weights used in the some of multiplications belong to different weight groups but correspond to a same piece of input data.
15. The convolution operation execution method according to claim 14 , including:
dividing the plurality of pieces of input data, and
supplying the divided pieces of input data to the PEs that perform the some of multiplications in parallel.
16. The non-transitory computer-readable storage medium storing a program according to claim 12 , causing the computer to execute:
controlling the plurality of PEs so as to perform in parallel some multiplications between each of the plurality of pieces of input data and each weight corresponding to each of the plurality of pieces of input data.
17. The non-transitory computer-readable storage medium storing a program according to claim 16 , causing the computer to execute:
controlling the plurality of PEs so as to execute in parallel the some multiplications between each of the plurality of pieces of input data and each of the plurality of weights, where the weights used in the some of multiplications belong to different weight groups but correspond to a same piece of input data.
18. The non-transitory computer-readable storage medium storing a program according to claim 16 , causing the computer to execute:
dividing the plurality of pieces of input data, and
supplying the divided pieces of input data to the PEs that perform the some of multiplications in parallel.
19. The convolution operation execution method according to claim 11 , further comprises:
causing the control part to acquire the notified order information; and
causing the control part to change associations between the pieces of input data and the plurality of weights included in the weight groups based on the order information.
20. The non-transitory computer-readable storage medium storing a program according to claim 12 , further comprises:
acquiring the notified order information; and
changing associations between the pieces of input data and the plurality of weights included in the weight groups based on the order information.