Sparsity-aware datastore for inference processing in deep neural network architectures

1. A computing system comprising:

a plurality of processing elements;

a memory to store compressed data associated with a plurality of tensors and a sparsity bitmap, wherein the compressed data is to be in a compressed format; and

a decode buffer that includes logic implemented at least partly in one or more of configurable or fixed-functionality hardware, the logic to:

prefetch the compressed data from the memory;

align the compressed data with the sparsity bitmap to generate decoded data; and

provide the decoded data to the plurality of processing elements.

2. The computing system of claim 1 , wherein the logic is to:

distribute the decoded data to the plurality of processing elements based on individual rates of consumption of the plurality of processing elements.

3. The computing system of claim 1 , wherein:

the decode buffer includes a plurality of datastores that include a plurality of buffers; and

wherein the logic is to store the compressed data in the plurality of datastores based on X values and Y values associated with the tensors.

4. The computing system of claim 1 , wherein:

the decode buffer includes a plurality of datastores that include a plurality of buffers, wherein the plurality of datastores includes a first group of datastores and a second group of datastores;

the logic includes a plurality of writing portions that are implemented at least partly in one or more of configurable or fixed-functionality hardware; and

the logic is to control a first writing portion of the plurality of writing portions to write to the first group of datastores, wherein the second group of datastores is inaccessible to the first writing portion.

5. The computing system of claim 1 , wherein:

the decode buffer includes a plurality of buffers;

the logic includes a writing portion that is implemented at least partly in one or more of configurable or fixed-functionality hardware; and

wherein the logic is to control the writing portion to access any of the plurality of buffers to execute write operations based on X and Y values associated with the plurality of tensors.

6. The computing system of claim 1 , wherein the logic is to:

align a first portion of the compressed data with a first portion of the sparsity bitmap; and

provide the first portion of the compressed data with the first portion of the sparsity bitmap to the plurality of processing elements to bypass a provision of unaligned portions of the compressed data and the sparsity bitmap to the processing elements.

7. A semiconductor apparatus comprising:

one or more substrates; and

logic coupled to the one or more substrates, wherein the logic is implemented in one or more of configurable or fixed-functionality hardware, the logic to:

prefetch compressed data and a sparsity bitmap from a memory to store the compressed data in a decode buffer, wherein the compressed data is associated with a plurality of tensors, wherein the compressed data is to be in a compressed format;

align the compressed data with the sparsity bitmap to generate decoded data; and

provide the decoded data to a plurality of processing elements.

8. The apparatus of claim 7 , wherein the logic is to:

distribute the decoded data to the processing elements based on individual rates of consumption of the processing elements.

9. The apparatus of claim 7 , wherein:

the decode buffer includes a plurality of datastores that include a plurality of buffers; and

the logic is to store the compressed data in the plurality of datastores based on X values and Y values associated with the tensors.

10. The apparatus of claim 7 , wherein:

the decode buffer includes a plurality of datastores that include a plurality of buffers, wherein the plurality of datastores includes a first group of datastores and a second group of datastores;

the logic includes a plurality of writing portions that are implemented at least partly in one or more of configurable or fixed-functionality hardware; and

the logic is to control a first writing portion of the plurality of writing portions to write to the first group of datastores, wherein the second group of datastores is inaccessible to the first writing portion.

11. The apparatus of claim 7 , wherein:

the decode buffer includes a plurality of buffers;

the logic includes a writing portion that is implemented at least partly in one or more of configurable or fixed-functionality hardware; and

wherein the logic is to is to control the writing portion to access any of the plurality of buffers to execute write operations based on X and Y values associated with the plurality of tensors.

12. The apparatus of claim 7 , wherein the logic is to:

align a first portion of the compressed data with a first portion of the sparsity bitmap; and

provide the first portion of the compressed data with the first portion of the sparsity bitmap to the plurality of processing elements to bypass a provision of unaligned portions of the compressed data and the sparsity bitmap to the processing elements.

13. The apparatus of claim 7 , wherein the logic includes transistor channel regions that are positioned within the one or more substrates.

14. At least one computer readable storage medium comprising a set of executable program instructions, which when executed by a computing system, cause the computing system to:

prefetch compressed data and a sparsity bitmap from a memory to store the compressed data in a decode buffer, wherein the compressed data is associated with a plurality of tensors, wherein the data is to be in a compressed format;

align the compressed data with the sparsity bitmap to generate decoded data; and

provide the decoded data to a plurality of processing elements.

15. The at least one computer readable storage medium of claim 14 , wherein the executable program instructions, when executed, further cause the computing system to:

distribute the decoded data to the processing elements based on individual rates of consumption of the processing elements.

16. The at least one computer readable storage medium of claim 14 , wherein:

the decode buffer includes a plurality of datastores that include a plurality of buffers; and

the executable program instructions, when executed, further cause the computing system to store the compressed data in the plurality of datastores based on X values and Y values associated with the tensors.

17. The at least one computer readable storage medium of claim 14 , wherein the executable program instructions, when executed, further cause the computing system to:

control a first writing portion of a plurality of writing portions to write to a first group of datastores of the decode buffer, wherein a second group of datastores of the decode buffer is inaccessible to the first writing portion, wherein the plurality of writing portions are implemented at least partly in one or more of configurable or fixed-functionality hardware.

18. The at least one computer readable storage medium of claim 14 ,

wherein the decode buffer includes a plurality of buffers; and

wherein the executable program instructions, when executed, further cause the computing system to control a writing portion, that is implemented at least partly in one or more of configurable or fixed-functionality hardware, to access any of the plurality of buffers to execute write operations based on X and Y values associated with the plurality of tensors.

19. The at least one computer readable storage medium of claim 14 , wherein the executable program instructions, when executed, further cause the computing system to:

align a first portion of the compressed data with a first portion of the sparsity bitmap; and

provide the first portion of the compressed data with the first portion of the sparsity bitmap to the plurality of processing elements to bypass a provision of unaligned portions of the compressed data and the sparsity bitmap to the processing elements.

20. A method comprising:

prefetching compressed data and a sparsity bitmap from a memory to store the compressed data in a decode buffer, wherein the compressed data is associated with a plurality of tensors, wherein the compressed data is in a compressed format;

aligning the compressed data with the sparsity bitmap to generate decoded data; and

providing the decoded data to a plurality of processing elements.

21. The method of claim 20 , further comprising:

distributing the decoded data to the processing elements based on individual rates of consumption of the processing elements.

22. The method of claim 20 , wherein:

the decode buffer includes a plurality of datastores that include a plurality of buffers; and

the method further includes storing the compressed data in the plurality of datastores based on X values and Y values associated with the tensors.

23. The method of claim 20 , further comprising:

controlling a first writing portion of a plurality of writing portions to write to a first group of datastores of the decode buffer, wherein a second group of datastores of the decode buffer is inaccessible to the first writing portion, wherein the plurality of writing portions is implemented at least partly in one or more of configurable or fixed-functionality hardware.

24. The method of claim 20 ,

wherein the decode buffer includes a plurality of buffers; and

wherein the method further comprises controlling a plurality of writing portions, that are implemented at least partly in one or more of configurable or fixed-functionality hardware, to access any of the plurality of buffers to execute write operations based on X and Y values associated with the plurality of tensors.

25. The method of claim 20 , the method further comprises:

aligning a first portion of the compressed data with a first portion of the sparsity bitmap; and

providing the first portion of the compressed data with the first portion of the sparsity bitmap to the plurality of processing elements to bypass a provision of unaligned portions of the compressed data and the sparsity bitmap to the processing elements.