Methods and apparatus to perform machine-learning model operations on sparse accelerators
Methods, apparatus, systems and articles of manufacture are disclosed to perform machine-learning model operations on sparse accelerators. An example apparatus includes first circuitry, second circuitry to generate sparsity data based on an acceleration operation, and third circuitry to instruct one or more data buffers to provide at least one of activation data or weight data based on the sparsity data to the first circuitry, the first circuitry to execute the acceleration operation based on the at least one of the activation data or the weight data.
1. An apparatus comprising:
first circuitry comprising one or more multiply accumulators;
a buffer to store first sparsity data, the first sparsity data indicating sparsity in input data of a first convolution type;
second circuitry to generate second sparsity data for a second convolution type;
one or more other buffers to store input data of a convolution; and
third circuitry to:
select sparsity data between the first sparsity data and the second sparsity data based on a type of a convolution,
identify one or more data elements in the input data of the convolution based on the selected sparsity data, and
transfer the one or more data elements from the one or more other data buffers to the first circuitry,
wherein the first circuitry is to execute at least part of the convolution by performing one or more multiply-accumulate operations on the one or more data elements.
2. The apparatus of claim 1 , wherein the second sparsity data includes a sparse weight vector, and the apparatus further comprises:
fourth circuitry to identify the type of the convolution based on configuration information, the sparse weight vector generated based on the configuration information,
wherein the third circuitry is further to generate a sparsity bit mask based on a sparse activation vector and the sparse weight vector and is to identify the one or more data elements by using the sparsity bit mask.
3. The apparatus of claim 1 , wherein the selected sparsity data includes at least one of activation sparsity data and weight sparsity data, the one or more data buffers include a weight data buffer and an activation data buffer, and the third circuitry is to:
generate a combined sparsity bit mask based on the activation sparsity data and the weight sparsity data;
identify one or more weights stored in the weight data buffer and one or more activations stored in the activation data buffer based on the combined sparsity bit mask;
transfer the one or more activations from the activation data buffer to the first circuitry; and
transfer the one or more weights from the weight data buffer to the first circuitry.
4. The apparatus of claim 1 , wherein the second sparsity data includes a sparse weight vector, the type of the convolution is depthwise convolution, and
the sparse weight vector includes a sequence of elements, in which an element has a different value from other elements.
5. The apparatus of claim 1 , wherein the second sparsity data includes a sparse weight vector, the type of the convolution is grouped convolution, and
the sparse weight vector includes a sequence of elements, in which a group of consecutive elements in the sequence has a different value from other elements in the sequence.
6. The apparatus of claim 1 , wherein the type of the convolution is elementwise addition, and wherein:
the second circuitry is to
generate the second sparsity data by generating a plurality of sparse weight vectors, each sparse weight vector including a non-zero element and one or more other zero elements.
7. The apparatus of claim 1 , wherein the second sparsity data includes a sparse weight matrix, the type of the convolution is dilated convolution, and
the second circuitry is to generate the sparse weight matrix based on a dimension of a kernel of the convolution and a dilation parameter of the convolution.
8. A non-transitory computer readable medium storing instructions executable to perform operations, the operations comprising:
storing first sparsity data, the first sparsity data indicating sparsity in input data of a first convolution type;
generating second sparsity data for a second convolution type;
selecting sparsity data between the first sparsity data and the second sparsity data based on a type of a convolution;
identifying one or more data elements in the input data of the convolution based on the selected sparsity data;
transferring the one or more data elements from one or more data buffers to one or more multiply accumulators; and
executing, by the one or more multiply accumulators, at least part of the convolution by performing one or more multiply-accumulate operations on the one or more data elements.
9. The non-transitory computer readable medium of claim 8 , wherein the second sparsity data includes a sparse weight vector, and the operations further comprise:
identifying the type of the convolution based on configuration information, the sparse weight vector generated based on the configuration information
generating a sparsity bit mask based on a sparse activation vector and the sparse weight vector,
wherein the one or more data elements are identified by using the sparsity bit mask.
10. The non-transitory computer readable medium of claim 8 , wherein the selected sparsity data includes at least one of activation sparsity data and weight sparsity data, the one or more data buffers include a weight data buffer and an activation data buffer, and the operations further comprise:
generating a combined sparsity bit mask based on the activation sparsity data and the weight sparsity data;
identifying one or more weights stored in the weight data buffer and one or more activations stored in the activation data buffer based on the combined sparsity bit mask;
transferring the one or more activations from the activation data buffer to the one or more multiply accumulators; and
transferring the one or more weights from the weight data buffer to the one or more multiply accumulators.
11. The non-transitory computer readable medium of claim 8 , wherein the second sparsity data includes a sparse weight vector, the type of the convolution is depthwise convolution, and
the sparse weight vector includes a sequence of elements, in which an element has a different value from other elements.
12. The non-transitory computer readable medium of claim 8 , wherein the second sparsity data includes a sparse weight vector, the type of the convolution a grouped convolution, and
the sparse weight vector includes a sequence of elements, in which a group of consecutive elements in the sequence has a different value from other elements in the sequence.
13. The non-transitory computer readable medium of claim 8 , wherein the type of the convolution is elementwise addition, and generating the second sparsity data comprises:
generating a plurality of sparse weight vectors, each sparse weight vector including a non-zero element and one or more other zero elements.
14. The non-transitory computer readable medium of claim 8 , wherein the second sparsity data includes a sparse weight matrix, the type of the convolution is a dilated convolution, and generating the second sparsity data comprises:
generating the sparse weight matrix based on a dimension of a kernel of the convolution and a dilation parameter of the convolution.
15. A method comprising:
storing first sparsity data, the first sparsity data indicating sparsity in input data of a first convolution type;
generating second sparsity data for a second convolution type;
selecting sparsity data between the first sparsity data and the second sparsity data based on a type of a convolution;
identifying one or more data elements in the input data of the convolution based on the selected sparsity data;
transferring the one or more data elements from one or more data buffers to one or more multiply accumulators; and
executing, by the one or more multiply accumulators, at least part of the convolution by performing one or more multiply-accumulate operations on the one or more data element.
16. The method of claim 15 , wherein the second sparsity data includes a sparse weight vector, and the method further comprises:
identifying the type of the convolution based on configuration information, the sparse weight vector generated based on the configuration information
generating a sparsity bit mask based on a sparse activation vector and the sparse weight vector,
wherein the one or more data elements are identified by using the sparsity bit mask.
17. The method of claim 15 , wherein the selected sparsity data includes at least one of activation sparsity data and weight sparsity data, the one or more data buffers include a weight data buffer and an activation data buffer, and the method further comprises:
generating a combined sparsity bit mask based on the activation sparsity data and the weight sparsity data;
identifying one or more weights stored in the weight data buffer and one or more activations stored in the activation data buffer based on the combined sparsity bit mask;
transferring the one or more activations from the activation data buffer to the one or more multiply accumulators; and
transferring the one or more weights from the weight data buffer to the one or more multiply accumulators.
18. The method of claim 15 , wherein the second sparsity data includes a sparse weight vector, the type of the convolution is depthwise convolution, and
the sparse weight vector includes a sequence of elements, in which an element has a different value from other elements.
19. The method of claim 15 , wherein the second sparsity data includes a sparse weight vector, the type of the convolution is grouped convolution, and
the sparse weight vector includes a sequence of elements, in which a group of consecutive elements in the sequence has a different value from other elements in the sequence.
20. The method of claim 15 , wherein the type of the convolution is elementwise addition, and the method further comprises:
generating a plurality of sparse weight vectors, each sparse weight vector including a non-zero element and one or more other zero elements.
21. The method of claim 15 , wherein the second sparsity data includes a sparse weight matrix, the type of the convolution is dilated convolution, and the method further comprises:
generating the sparse weight matrix based on a dimension of a kernel of the convolution and a dilation parameter of the convolution.