Spatially sparse neural network accelerator for multi-dimension visual analytics
Systems, apparatuses and methods may provide for technology that decodes data via an instruction that indicates a number of rulebooks to be processed, an input feature size, an output feature size, and a plurality of feature map base addresses, rearranges spatially distributed voxel output feature maps in the decoded data based on weight planes, and performs a channel-wise multiply-accumulate (MAC) operation on the rearranged spatially distributed voxel output feature maps to obtain an output, wherein the channel-wise MAC operation is performed as partial accumulations by a plurality of processing elements.
1. A computing system comprising:
a network controller;
a processor coupled to the network controller; and
a memory coupled to the processor, the memory including a set of executable program instructions, which when executed by the processor, cause the computing system to:
decode data via an instruction that indicates a number of rulebooks to be processed, an input feature size, an output feature size, and a plurality of feature map base addresses,
rearrange spatially distributed voxel output feature maps in the decoded data based on weight planes, and
perform a channel-wise multiply-accumulate (MAC) operation on the rearranged spatially distributed voxel output feature maps to obtain an output, wherein the channel-wise MAC operation is performed as partial accumulations by a plurality of processing elements in the processor.
2. The computing system of claim 1 , wherein the set of executable program instructions, when executed, further cause the computing system to select a rulebook type based on application information, and wherein the rulebook type is one of an input-to-output type or an output-to-input type.
3. The computing system of claim 2 , wherein the set of executable program instructions, when executed, further cause the computing system to read the data from a level one (L1) cache, wherein the data has a variable number of dimensions greater than two, and wherein the data is in a rulebook line format associated with the rulebook type.
4. The computing system of claim 2 , wherein the set of executable program instructions, when executed, further cause the computing system to interchange output data from an index queue based on the rulebook type.
5. The computing system of claim 1 , wherein the set of executable program instructions, when executed, further cause the computing system to allocate memory and the plurality of processing elements based on a sparsity of the data, and wherein the channel-wise MAC operation is to identify overlapping output feature maps and merge the overlapping output feature maps locally.
6. The computing system of claim 1 , further including a static buffer and a data storage, wherein the set of executable program instructions, when executed, further cause the computing system to:
store feature mapping pair information to the static buffer; and
store index information corresponding to the feature mapping pair information to a location in the data storage.
7. A semiconductor apparatus comprising:
one or more substrates; and
logic coupled to the one or more substrates, wherein the logic is implemented at least partly in one or more of configurable logic or fixed-functionality logic hardware, the logic coupled to the one or more substrates to:
decode data via an instruction that indicates a number of rulebooks to be processed, an input feature size, an output feature size, and a plurality of feature map base addresses;
rearrange spatially distributed voxel output feature maps in the decoded data based on weight planes; and
perform a channel-wise multiply-accumulate (MAC) operation on the rearranged spatially distributed voxel output feature maps to obtain an output, wherein the channel-wise MAC operation is performed as partial accumulations by a plurality of processing elements in the logic coupled to the one or more substrates.
8. The apparatus of claim 7 , wherein the logic coupled to the one or more substrates is to select a rulebook type based on application information, wherein the rulebook type is one of an input-to-output type or an output-to-input type.
9. The apparatus of claim 8 , wherein the logic coupled to the one or more substrates is to read the data from a level one (L1) cache, wherein the data has a variable number of dimensions greater than two, and wherein the data is in a rulebook line format associated with the rulebook type.
10. The apparatus of claim 8 , wherein the logic coupled to the one or more substrates is to interchange output data from an index queue based on the rulebook type.
11. The apparatus of claim 7 , wherein the logic coupled to the one or more substrates is to allocate memory and the plurality of processing elements based on a sparsity of the data, and wherein the channel-wise MAC operation is to identify overlapping output feature maps and merge the overlapping output feature maps locally.
12. The apparatus of claim 7 , wherein the logic coupled to the one or more substrates is to:
store feature mapping pair information to a static buffer; and
store index information corresponding to the feature mapping pair information to a data storage location.
13. The apparatus of claim 7 , wherein the logic coupled to the one or more substrates includes transistor channel regions that are positioned within the one or more substrates.
14. At least one non-transitory computer readable storage medium comprising a set of executable program instructions, which when executed by a computing system, cause the computing system to:
decode data via an instruction that indicates a number of rulebooks to be processed, an input feature size, an output feature size, and a plurality of feature map base addresses;
rearrange spatially distributed voxel output feature maps in the decoded data based on weight planes; and
perform a channel-wise multiply-accumulate (MAC) operation on the rearranged spatially distributed voxel output feature maps to obtain an output, wherein the channel-wise MAC operation is performed as partial accumulations by a plurality of processing elements in the computing system.
15. The at least one non-transitory computer readable storage medium of claim 14 , wherein the set of executable program instructions, when executed, further cause the computing system to select a rulebook type based on application information, and wherein the rulebook type is one of an input-to-output type or an output-to-input type.
16. The at least one non-transitory computer readable storage medium of claim 15 , wherein the set of executable program instructions, when executed, further cause the computing system to read the data from a level one (L1) cache, wherein the data has a variable number of dimensions greater than two, and wherein the data is in a rulebook line format associated with the rulebook type.
17. The at least one non-transitory computer readable storage medium of claim 15 , wherein the set of executable program instructions, when executed, further cause the computing system to interchange output data from an index queue based on the rulebook type.
18. The at least one non-transitory computer readable storage medium of claim 14 , wherein the set of executable program instructions, when executed, further cause the computing system to allocate memory and the plurality of processing elements based on a sparsity of the data, and wherein the channel-wise MAC operation is to identify overlapping output feature maps and merge the overlapping output feature maps locally.
19. The at least one non-transitory computer readable storage medium of claim 14 , wherein the set of executable program instructions, when executed, further cause the computing system to:
store feature mapping pair information to a static buffer; and
store index information corresponding to the feature mapping pair information to a data storage location.
20. A method comprising:
decoding data via an instruction that indicates a number of rulebooks to be processed, an input feature size, an output feature size, and a plurality of feature map base addresses;
rearranging spatially distributed voxel output feature maps in the decoded data based on weight planes; and
performing a channel-wise multiply-accumulate (MAC) operation on the rearranged spatially distributed voxel output feature maps to obtain an output, wherein the channel-wise MAC operation is performed as partial accumulations by a plurality of processing elements.
21. The method of claim 20 , further including selecting a rulebook type based on application information, wherein the rulebook type is one of an input-to-output type or an output-to-input type.
22. The method of claim 21 , further including reading the data from a level one (L1) cache, wherein the data has a variable number of dimensions greater than two, and wherein the data is in a rulebook line format associated with the rulebook type.
23. The method of claim 21 , further including interchanging output data from an index queue based on the rulebook type.
24. The method of claim 20 , further including allocating memory and the plurality of processing elements based on a sparsity of the data, and wherein the channel-wise MAC operation identifies overlapping output feature maps and merges the overlapping output feature maps locally.
25. The method of claim 20 , further including:
storing feature mapping pair information to a static buffer; and
storing index information corresponding to the feature mapping pair information to a data storage location.