Smart memory handling and data management for machine learning networks
A mechanism is described for facilitating memory handling and data management in machine learning at autonomous machines. A method of embodiments, as described herein, includes detecting multiple tables associated with multiple neural networks at multiple autonomous machines, where each of the multiple tables include an index. The method may further include combining the multiple tables and multiple indexes associated with the multiple tables into a single table and a single index, respectively, where the single table is communicated to the multiple autonomous machines to allow simultaneous processing of one or more portions of the single table using one or more memory devices and one or more processors of one or more of the multiple autonomous machines.
1 . An apparatus comprising:
one or more processors to process data for operation of the apparatus, the one or more processors including one or more graphical processing units (GPUs); and
a memory for storage of data, the memory including a memory address portion;
wherein the one or more processors are to perform processing with a neural network, including the one or more processors to:
generate a plurality of memory addresses for storage in the memory address portion, the plurality of memory addresses being based at least in part on a memory pattern associated with the neural network;
rearrange a plurality of feature maps for the neural network such that each feature map starts from an address of the plurality of memory addresses; and
fetch the plurality of memory addresses to load the plurality of feature maps for the one or more processors.
2 . The apparatus of claim 1 , wherein the one or more processors are further to:
convolve the plurality of feature maps with a specified kernel; and
accumulate results of each convolved feature map to generate a combined feature map.
3 . The apparatus of claim 1 , wherein the plurality of memory addresses provide memory layouts that are relatively sparse for two-dimensional (2D) data and relatively dense for three-dimensional (3D) data.
4 . The apparatus of claim 1 , wherein the rearrangement of the plurality of feature maps is performed at one or more of a memory level, a cache level, or a register level.
5 . The apparatus of claim 1 , wherein fetching the plurality of memory addresses includes the one or more processors to fetch the plurality of memory addresses in parallel.
6 . The apparatus of claim 1 , wherein the neural network is a convolutional neural network (CNN).
7 . The apparatus of claim 1 , wherein the one or more GPUs include circuitry for processing of a neural network.
8 . The apparatus of claim 7 , wherein the circuitry for processing of the neural network includes circuitry to perform memory layout and circuitry to perform feature matching.
9 . The apparatus of claim 7 , wherein the circuitry for processing of the neural network includes circuitry based on the memory pattern associated with the neural network.
10 . A method comprising:
generating a plurality of memory addresses for storage in a memory address portion of a memory, the plurality of memory addresses being based at least in part on a memory pattern associated with a neural network;
rearranging a plurality of feature maps for the neural network such that each feature map starts from an address of the plurality of memory addresses; and
fetching the plurality of memory addresses to load the plurality of feature maps for processing by one or more processors.
11 . The method of claim 10 , further comprising:
convolving the plurality of feature maps with a specified kernel; and
accumulating results of each convolved feature map to generate a combined feature map.
12 . The method of claim 10 , wherein the plurality of memory addresses provide memory layouts that are relatively sparse for two-dimensional (2D) data and relatively dense for three-dimensional (3D) data.
13 . The method of claim 10 , wherein the rearrangement of the plurality of feature maps is performed at one or more of a memory level, a cache level, or a register level.
14 . The method of claim 10 , wherein fetching the plurality of memory addresses includes fetching the plurality of memory addresses in parallel.
15 . The method of claim 10 , wherein the neural network is a convolutional neural network (CNN).
16 . At least one non-transitory machine-readable medium comprising instructions that when executed by a computing device, cause the computing device to perform operations comprising: generating a plurality of memory addresses for storage in a memory address portion of a memory, the plurality of memory addresses being based at least in part on a memory pattern associated with a neural network; rearranging a plurality of feature maps for the neural network such that each feature map starts from an address of the plurality of memory addresses; and fetching the plurality of memory addresses to load the plurality of feature maps for processing by one or more processors.
17 . The non-transitory machine-readable medium of claim 16 , wherein the operations further comprise: convolving the plurality of feature maps with a specified kernel; and accumulating results of each convolved feature map to generate a combined feature map.
18 . The non-transitory machine-readable medium of claim 16 , wherein the plurality of memory addresses provide memory layouts that are relatively sparse for two-dimensional (2D) data and relatively dense for three-dimensional (3D) data.
19 . The non-transitory machine-readable medium of claim 16 , wherein the rearrangement of the plurality of feature maps is performed at one or more of a memory level, a cache level, or a register level.
20 . The non-transitory machine-readable medium of claim 16 , wherein fetching the plurality of memory addresses includes fetching the plurality of memory addresses in parallel.