Computation unit, related apparatus, and method
This disclosure provides a computation unit, a related apparatus, and a method. The computation unit includes: a weight buffer adapted to store a row vector fetched from an M×Kα sparsified weight matrix, where M and K are respectively a number of rows and a number of columns of the weight matrix before being sparsified, and α is a sparsity coefficient; an excitation buffer adapted to store a K×N excitation matrix; an index selector adapted to store a selection index corresponding to the row vector, and select a row of the excitation matrix based on the selection index, to obtain a Kα×N selected excitation matrix; and a dot product computation unit adapted to multiply the row vector by the selected excitation matrix. This disclosure implements a manner of running a DNN on hardware. In such a manner, structured sparsity of a DNN can be fully utilized, so that inference efficiency is improved; moreover, a register file occupies relatively small bandwidth, and a timing constraint is weak.
1 . A computation unit, comprising:
a weight buffer adapted to store a row vector fetched from an M×Kα sparsified weight matrix, wherein M and K are respectively a number of rows and a number of columns of the weight matrix before being sparsified, and a is a sparsity coefficient;
an excitation buffer adapted to store a K×N excitation matrix, where K and N are respectively a number of rows and a number of columns of the excitation matrix;
an index selector adapted to store a selection index corresponding to the row vector, and select a row of the excitation matrix based on the selection index, to obtain a Kα×N selected excitation matrix; and
a dot product computation unit adapted to multiply the row vector by the selected excitation matrix.
2 . The computation unit according to claim 1 , wherein there are L weight buffers for respectively storing L row vectors that are sequentially fetched from the sparsified weight matrix, there are L index selectors for respectively storing selection indexes corresponding to the L row vectors, and there are L·N dot product computation units for respectively multiplying the L row vectors by N column vectors in the selected excitation matrix.
3 . The computation unit according to claim 2 , wherein L=2.
4 . The computation unit according to claim 2 , wherein the dot product computation unit sets M/L clock cycles for multiplication of the weight matrix and the excitation matrix, wherein M is an integer multiple of L; and in a single clock cycle, the row vector in the weight buffer is multiplied by the N column vectors in the selected excitation matrix by using N dot product computation units corresponding to the row vector.
5 . The computation unit according to claim 2 , wherein the row vector is loaded to the weight buffer a first predetermined number of clock cycles before a clock cycle in which the dot product computation unit performs a multiplication operation on the row vector.
6 . The computation unit according to claim 5 , wherein the first predetermined number is 1.
7 . The computation unit according to claim 2 , wherein the selection index is loaded to the index selector a second predetermined number of clock cycles before a clock cycle in which the dot product computation unit performs a multiplication operation on the selected excitation matrix obtained through selection based on the selection index.
8 . The computation unit according to claim 7 , wherein the second predetermined number is 1.
9 . The computation unit according to claim 2 , wherein the excitation matrix is loaded to the excitation buffer a third predetermined number of clock cycles before a clock cycle in which the dot product computation unit performs a multiplication operation on the first selected excitation matrix obtained through selection from the excitation matrix, and is retained for M/L clock cycles.
10 . The computation unit according to claim 9 , wherein the third predetermined number is M/L+1.
11 . The computation unit according to claim 9 , further comprising:
a candidate excitation buffer adapted to store a next excitation matrix, wherein once the excitation matrix is unloaded from the excitation buffer, the next excitation matrix stored in the candidate excitation buffer is loaded to the excitation buffer.
12 . The computation unit according to claim 1 , wherein the excitation buffer comprises a first excitation buffer and a second excitation buffer that serve as ping-pong buffers, respectively adapted to store a current excitation matrix and a next excitation matrix; wherein the first excitation buffer and the second excitation buffer are both connected to the index selector; and after the dot product computation unit completes multiplication of each row vector in the sparsified weight matrix and a selected current excitation matrix, the index selector switches to selection in the second excitation buffer.
13 . A deep neural network running method, comprising:
fetching a row vector from an M×Kα sparsified weight matrix, and placing the row vector into a weight buffer, wherein M and K are respectively a number of rows and a number of columns of the weight matrix before being sparsified, and α is a sparsity coefficient;
placing a K×N excitation matrix into an excitation buffer, where K and N are respectively a number of rows and a number of columns of the excitation matrix;
placing a selection index corresponding to the row vector into an index selector;
performing selection on the excitation matrix based on the selection index by using the index selector, to obtain a Kα×N selected excitation matrix; and
multiplying the row vector by the selected excitation matrix by using a dot product computation unit.
14 . The method according to claim 13 , wherein there are L weight buffers; and the fetching a row vector from an M×Kα sparsified weight matrix, and placing the row vector into a weight buffer comprises: sequentially fetching L row vectors from the sparsified weight matrix, and placing the L row vectors into the L weight buffers respectively.
15 . The method according to claim 14 , wherein there are L index selectors; and the placing a selection index corresponding to the row vector into an index selector comprises: placing selection indexes corresponding to the L row vectors into the L index selectors.
16 . The method according to claim 14 , wherein there are L·N dot product computation units; and the multiplying the row vector by the selected excitation matrix by using a dot product computation unit comprises: multiplying the L row vectors by N column vectors in the selected excitation matrix by using the L·N dot product computation units respectively.
17 . The method according to claim 16 , wherein the multiplying the row vector by the selected excitation matrix comprises:
setting M/L clock cycles for multiplication of the weight matrix and the excitation matrix, wherein M is an integer multiple of L; and in a single clock cycle, multiplying the row vector in the weight buffer by the N column vectors in the selected excitation matrix by using N dot product computation units corresponding to the row vector.
18 . The method according to claim 17 , wherein the fetching a row vector from an M×Kα sparsified weight matrix, and placing the row vector into a weight buffer comprises: placing the fetched row vector into the weight buffer a first predetermined number of clock cycles before a clock cycle in which the dot product computation unit performs a multiplication operation on the row vector.
19 . The method according to claim 17 , wherein the placing a selection index corresponding to the row vector into an index selector comprises:
placing the selection index into the index selector a second predetermined number of clock cycles before a clock cycle in which the dot product computation unit performs a multiplication operation on the selected excitation matrix obtained through selection based on the selection index.
20 . The method according to claim 17 , wherein the placing a K×N excitation matrix into an excitation buffer comprises:
placing the excitation matrix into the excitation buffer a third predetermined number of clock cycles before a clock cycle in which the dot product computation unit performs a multiplication operation on the first selected excitation matrix obtained through selection from the excitation matrix, and retaining the excitation matrix for M/L clock cycles.