Systems and methods for reading and writing sparse data in a neural network accelerator
Disclosed herein includes a system, a method, and a device for reading and writing sparse data in a neural network accelerator. A plurality of slices can be established to access a memory having an access size of a data word. A first slice can be configured to access a first side of the data word in memory. Circuitry can access a mask identifying byte positions within the data word having non-zero values. The circuitry can modify the data word to have non-zero byte values stored starting at an end of the first side, and any zero byte values stored in a remainder of the data word. A determination can be made whether a number of non-zero byte values is less than or equal to a first access size of the first slice. The circuitry can write the modified data word to the memory via at least the first slice.
1. A method comprising:
establishing, by circuitry, a plurality of slices to access a memory having an access size of a data word, a first slice of the plurality of slices configured to access a first side of the data word in memory, the first side having a size less than that of the data word;
accessing, by the circuitry, a mask identifying byte positions within the data word having non-zero values, wherein each bit of the mask has a first value or a second value, the first value indicating that a byte of the data word corresponds to a non-zero byte value, the second value indicating that the byte of the data word corresponds to a zero byte value;
modifying, by the circuitry, the data word to have non-zero byte values stored starting at an end of the first side, and any zero byte values stored in a remainder of the data word;
determining, by the circuitry, whether non-zero byte data within the data word corresponding to first values in the mask is less than or equal to a first access size of the first slice; and
writing, by the circuitry responsive to the determination, the modified data word to the memory via at least the first slice.
2. The method of claim 1 , further comprising generating the mask as one of a one-byte mask or a two-byte mask.
3. The method of claim 1 , further comprising determining, by the circuitry, the number of non-zero byte values of the first side is less than or equal to the first access size of the first slice.
4. The method of claim 3 , further comprising writing, by the circuitry, the first side of the modified data word only via the first slice.
5. The method of claim 1 , further comprising determining, by the circuitry, the number of non-zero byte values starting at the end of the first side is greater than the first access size of the first slice.
6. The method of claim 5 , further comprising writing, by the circuitry, the bytes of the first side of the modified data word via the first slice and remainder non-zero bytes via a second slice.
7. The method of claim 1 , wherein one of the first access size of the first slice or a second access size of a second slice of the plurality of slices is selected to provide a predetermined reduction in power usage of the circuitry in accessing the memory.
8. The method of claim 1 , further comprising determining, by the circuitry, the number of non-zero byte values of the modified data word based on the mask.
9. A method comprising:
establishing, by circuitry, a plurality of slices to access a memory having an access size of a data word, a first slice of the plurality of slices configured to access a first side of the data word in the memory, the first side having a size less than that of the data word;
accessing, by the circuitry, a mask identifying positions within the data word having non-zero values, wherein each bit of the mask has a first value or a second value, the first value indicating that a byte of the data word corresponds to a non-zero byte value, the second value indicating that the byte of the data word corresponds to a zero byte value;
determining, by the circuitry, whether non-zero byte data within the data word corresponding to first values in the mask is less than or equal to a first access size of the first slice;
reading, by the circuitry responsive to the determination, a packed representation of the data word stored in memory via the first slice, the packed representation comprising the non-zero byte values stored starting at an end of the first side of the data word and any zero byte values stored in a remainder of the data word; and
providing, by the circuitry from the packed representation, the data word having the non-zero byte values and the zero byte values at the positions indicated by the mask.
10. The method of claim 9 , wherein the mask identifies the position of non-zero byte values on a one-byte basis or a two-byte basis.
11. The method of claim 9 , further comprising determining, by the circuitry, the number of non-zero byte values of the data word is less than or equal to the first access size of the first slice.
12. The method of claim 11 , further comprising reading, by the circuitry, the packed representation of the data word via only the first slice.
13. The method of claim 9 , further comprising determining, by the circuitry, the number of non-zero byte values of the data word is greater than the first access size of the first slice.
14. The method of claim 13 , further comprising reading, by the circuitry, a first side of the data word via the first slice and any remainder non-zero bytes via a second slice of the plurality of slices.
15. The method of claim 1 , wherein one of the first access size of the first slice or a second access size of a second slice of the plurality of slices is selected to provide a predetermined reduction in power usage of the circuitry in reading from the memory.
16. A device comprising:
memory having an access size of a data word; and
circuitry configured to:
establish a plurality of slices to access the memory, a first slice of the plurality of slices configured to access a left side of a data word in the memory;
access a mask identifying byte positions within the data word having non-zero values, wherein each bit of the mask has a first value or a second value, the first value indicating that a byte of the data word corresponds to a non-zero byte value, the second value indicating that the byte of the data word corresponds to a zero byte value;
modify the data word to have non-zero byte values stored starting at an end of a first side of the data word and any zero byte values stored in a remainder of the data word;
determine whether non-zero byte data within the data word corresponding to first values in the mask is less than or equal to a first access size of the first slice; and
write, responsive to the determination, the modified data word to the memory via at least the first slice.
17. The device of claim 16 , wherein the circuitry is configured to read, a packed representation of the data word stored in memory via at least the first slice, wherein the packed representation comprises the non-zero byte values stored in the first side of the data word and any zero byte values stored in a remaining bytes of the data word.
18. The device of claim 17 , wherein the circuitry is configured to create the data word to have the non-zero byte values and the zero byte values stored in the positions indicated by the mask.
19. The device of claim 16 , wherein the circuitry is further configured to read the first side of the data word via the first slice and any remaining non-zero bytes via a second slice of the plurality of slices.
20. The device of claim 16 , wherein one of the first access size of the first slice or a second access size of a second slice of the plurality of slices is selected to provide a predetermined reduction in power usage of the circuitry in reading from the memory.