Methods, systems, articles of manufacture, and apparatus to decode zero-value-compression data vectors
Methods, systems, articles of manufacture, and apparatus are disclosed to decode zero-value-compression data vectors. An example apparatus includes: a buffer monitor to monitor a buffer for a header including a value indicative of compressed data; a data controller to, when the buffer includes compressed data, determine a first value of a sparse select signal based on (1) a select signal and (2) a first position in a sparsity bitmap, the first value of the sparse select signal corresponding to a processing element that is to process a portion of the compressed data; and a write controller to, when the buffer includes compressed data, determine a second value of a write enable signal based on (1) the select signal and (2) a second position in the sparsity bitmap, the second value of the write enable signal corresponding to the processing element that is to process the portion of the compressed data.
1. An apparatus for performing a multiply-accumulate (MAC) operation, the apparatus comprising:
a memory to:
store a compressed tensor, the compressed tensor comprising one or more nonzero-valued elements in a tensor associated with the MAC operation, the tensor associated with the MAC operation further comprises one or more zero-valued elements, and
store a sparsity bitmap, the sparsity bitmap encoding one or more positions of the one or more nonzero-valued elements in the tensor associated with the MAC operation;
a processing element to perform the MAC operation using a nonzero-valued element stored in the memory; and
a multiplexer associated with the processing element, the multiplexer to:
receive a signal generated based on the sparsity bitmap,
select, based on the signal, the nonzero-valued element from the one or more non-zero elements, and
transmit the nonzero-valued element from the memory to the processing element.
2. The apparatus of claim 1 , further comprising:
an additional processing element to perform the MAC operation using an additional nonzero-valued element stored in the memory; and
an additional multiplexer associated with the additional processing element, the additional multiplexer to:
receive an additional signal generated based on the sparsity bitmap,
select, based on the additional signal, the additional nonzero-valued element from the one or more nonzero-valued elements, and
transmit the additional nonzero-valued element from the memory to the additional processing element.
3. The apparatus of claim 1 , wherein the tensor associated with the MAC operation is an activation tensor.
4. The apparatus of claim 3 , further comprising:
another memory to:
store a compressed weight tensor, the compressed weight tensor comprising one or more nonzero-valued weights in a weight tensor, the weight tensor further comprises one or more zero-valued weights, and
store a weight sparsity bitmap, the weight sparsity bitmap encoding one or more positions of the one or more nonzero-valued weights in the weight tensor,
wherein:
a nonzero-valued weight is selected from the one or more nonzero-valued weights based on the weight sparsity bitmap and transmitted to the processing element, and
the processing element is to perform the MAC operation using the nonzero-valued weight.
5. The apparatus of claim 1 , wherein the tensor associated with the MAC operation is a weight tensor.
6. The apparatus of claim 5 , further comprising:
another memory to:
store a compressed activation tensor, the compressed activation tensor comprising one or more nonzero-valued activations in an activation tensor, the activation tensor further comprises one or more zero-valued activations, and
store an activation sparsity bitmap, the activation sparsity bitmap encoding one or more positions of the one or more nonzero-valued activation in the activation tensor;
wherein:
a nonzero-valued activation is selected from the one or more nonzero-valued activations based on the activation sparsity bitmap and transmitted to the processing element, and
the processing element is to perform the MAC operation using the nonzero-valued activation.
7. The apparatus of claim 1 , wherein the one or more nonzero-valued elements are stored at consecutive memory addresses of the memory.
8. The apparatus of claim 1 , wherein the sparsity bitmap and the compressed tensor are stored at consecutive memory addresses.
9. The apparatus of claim 1 , wherein the MAC operation is further associated with another tensor, and the another tensor comprises a plurality of other elements.
10. The apparatus of claim 1 , wherein the one or more nonzero-valued elements and the one or more zero-valued elements are in different channels.
11. A method for performing a multiply-accumulate (MAC) operation, the method comprising:
storing a compressed tensor in a memory, the compressed tensor comprising one or more nonzero-valued elements in a tensor associated with the MAC operation, the tensor associated with the MAC operation further comprises one or more zero-valued elements;
storing a sparsity bitmap in the memory, the sparsity bitmap encoding one or more positions of the one or more nonzero-valued elements in the tensor associated with the MAC operation;
providing a signal to a multiplexer, wherein the signal corresponds to a nonzero-valued element in the memory and is generated using the sparsity bitmap, and the multiplexer is associated with a processing element; and
transmitting, by the multiplexer based on the signal, the nonzero-valued element from the memory to the processing element, wherein the processing element is to perform the MAC operation using the nonzero-valued element.
12. The method of claim 11 , further comprising:
providing an additional signal to an additional multiplexer, wherein the additional signal corresponds to an additional nonzero-valued element in the memory and is generated using the sparsity bitmap, and the additional multiplexer is associated with an additional processing element; and
transmitting, by the additional multiplexer based on the additional signal, the additional nonzero-valued element from the memory to the additional processing element, wherein the additional processing element is to perform the MAC operation using the additional nonzero-valued element.
13. The method of claim 11 , further comprising:
storing a compressed weight tensor in another memory, the compressed weight tensor comprising one or more nonzero-valued weights in a weight tensor, the weight tensor further comprises one or more zero-valued weights; and
storing a weight sparsity bitmap in the another memory, the weight sparsity bitmap encoding one or more positions of the one or more nonzero-valued weights in the weight tensor,
wherein:
a nonzero-valued weight is selected from the one or more nonzero-valued weights based on the weight sparsity bitmap and transmitted to the processing element,
the processing element is to perform the MAC operation using the nonzero-valued weight, and
the tensor associated with the MAC operation is an activation tensor.
14. The method of claim 11 , further comprising:
storing a compressed activation tensor in another memory, the compressed activation tensor comprising one or more nonzero-valued activations in an activation tensor, the activation tensor further comprises one or more zero-valued activations; and
storing an activation sparsity bitmap in the another memory, the activation sparsity bitmap encoding one or more positions of the one or more nonzero-valued activation in the activation tensor,
wherein:
a nonzero-valued activation is selected from the one or more nonzero-valued activations based on the activation sparsity bitmap and transmitted to the processing element,
the processing element is to perform the MAC operation using the nonzero-valued activation, and
the tensor associated with the MAC operation is a weight tensor.
15. The method of claim 11 , wherein the one or more nonzero-valued elements are stored at consecutive memory addresses of the memory.
16. The method of claim 11 , wherein the sparsity bitmap and the compressed tensor are stored at consecutive memory addresses.
17. The method of claim 11 , wherein the MAC operation is further associated with another tensor, and the another tensor comprises a plurality of other elements.
18. The method of claim 11 , wherein the one or more nonzero-valued elements and the one or more zero-valued elements are in different channels.
19. One or more non-transitory computer-readable media storing instructions executable to perform operations for performing a multiply-accumulate (MAC) operation, the operations comprising:
storing a compressed tensor in a memory, the compressed tensor comprising one or more nonzero-valued elements in a tensor associated with the MAC operation, the tensor associated with the MAC operation further comprises one or more zero-valued elements;
storing a sparsity bitmap in the memory, the sparsity bitmap encoding one or more positions of the one or more nonzero-valued elements in the tensor associated with the MAC operation;
providing a signal to a multiplexer, wherein the signal corresponds to a nonzero-valued element in the memory and is generated using the sparsity bitmap, and the multiplexer is associated with a processing element; and
transmitting, by the multiplexer based on the signal, the nonzero-valued element from the memory to the processing element, wherein the processing element is to perform the MAC operation using the nonzero-valued element.
20. The one or more non-transitory computer-readable media of claim 19 , wherein the operations further comprise:
storing a compressed weight tensor in another memory, the compressed weight tensor comprising one or more nonzero-valued weights in a weight tensor, the weight tensor further comprises one or more zero-valued weights; and
storing a weight sparsity bitmap in the another memory, the weight sparsity bitmap encoding one or more positions of the one or more nonzero-valued weights in the weight tensor,
wherein:
a nonzero-valued weight is selected from the one or more nonzero-valued weights based on the weight sparsity bitmap and transmitted to the processing element,
the processing element is to perform the MAC operation using the nonzero-valued weight, and
the tensor associated with the MAC operation is an activation tensor.