Optimized neural network data organization
In some implementations, the present disclosure relates to a method. The method includes obtaining a set of weights for a neural network comprising a plurality of nodes and a plurality of connections between the plurality of nodes. The method also includes identifying a first subset of weights and a second subset of weights based on the set of weights. The first subset of weights comprises weights that used by the neural network. The second subset of weights comprises weights that are prunable. The method further includes storing the first subset of weights in a first portion of a memory. A first error correction code is used for the first portion of the memory. The method further includes storing the second subset of weights in a second portion of the memory. A second error correction code is used for the second portion of the memory. The second error correction code is weaker than the first error correction code.
1. A method, comprising:
organizing various types of data into two or more hierarchies, such that data in different hierarchies is stored in different portions of memory comprising a plurality of dies; and
storing weight data comprising a first and second subset of weights for a neural network, wherein:
the first subset of weights for the neural network is stored in a first portion of the memory, the neural network comprising a plurality of nodes and a plurality of connections between the plurality of nodes, wherein:
the first subset of weights comprises unprunable weights used by the neural network; and
a first error correction code is used for the first portion of the memory; and
the second subset of weights for the neural network is stored in a second portion of the memory, wherein:
the second subset of weights comprises prunable weights; and
a second error correction code is used for the second portion of the memory,
wherein the second error correction code is weaker than the first error correction code; and
wherein copies of the weight data are duplicated across the plurality of dies.
2. The method of claim 1 , wherein more overprovisioning is utilized in the second portion of the memory than in the first portion of the memory.
3. The method of claim 1 , wherein storing the first subset of weights in the first portion of the memory comprises:
logically arranging the first subset of weights into a bit array, wherein:
each row of the bit array comprises a weight of the first subset of weights; and
the first subset of weights are aligned by bit position.
4. The method of claim 3 , wherein storing the first subset of weights in the first portion of the memory further comprises:
encoding columns of the bit array using different error correction codes, wherein the different error correction codes comprise the first error correction code.
5. The method of claim 4 , wherein:
each column of the bit array is associated with a different bit significance; and
columns associated with a lower bit significance are encoded using weaker error correction codes than columns associated with a higher bit significance.
6. The method of claim 5 , wherein bits in each column of the bit array columns associated with lower bit significances are updated more frequently than bits associated in columns associated with higher bit significances.
7. The method of claim 1 , wherein:
the memory comprises multiple types; and
the first portion of the memory and the second portion of the memory are of a first type of memory.
8. The method of claim 7 , wherein:
file metadata associated with input files and inference results obtained based on the input files, are stored in a third portion of the memory;
the input files are stored in a fourth portion of the memory; and
the fourth portion of the memory is of a second type of memory.
9. The method of claim 8 , wherein:
the first portion of the memory, the second portion of the memory, the third portion of the memory, and the fourth portion of the memory use different error correction codes and different amounts of overprovisioning; and
the first portion of the memory, the second portion of the memory, the third portion of the memory, and the fourth portion of the memory have different access latencies and different reliabilities.
10. A data storage device, comprising:
a memory configured to store data; and
a controller coupled to the memory, the controller configured to:
organize various types of data for a neural network into two or more hierarchies, such that data in different hierarchies is stored in different portions of the memory comprising a plurality of dies; and
selectively store weight data comprising a first and second subset of weights for the neural network, wherein:
the first subset of weights for the neural network is stored in a first portion of the memory, the neural network comprising a plurality of nodes and a plurality of connections between the plurality of nodes, wherein:
the first subset of weights comprises weights used by the neural network; and
a first error correction code is used for the first portion of the memory; and
the second subset of weights for the neural network is stored in a second portion of the memory, wherein:
the second subset of weights comprises prunable weights that, once removed, reduce an amount of data used to store weights for the neural network;
a second error correction code is used for the second portion of the memory; and
the second error correction code is weaker than the first error correction code;
wherein copies of the weight data are duplicated across the plurality of dies.
11. The data storage device of claim 10 , wherein more overprovisioning is used in the second portion of the memory than is used in the first portion of the memory.
12. The data storage device of claim 10 , wherein to selectively store the first subset of weights in the first portion of the memory, the controller is further configured to:
logically arrange the first subset of weights into a bit array, wherein:
each row of the bit array comprises a weight of the first subset of weights; and
the first subset of weights are aligned by bit position.
13. The data storage device of claim 12 , wherein to selectively store the first subset of weights in the first portion of the memory, the controller is further configured to:
encode columns of the bit array using different error correction codes, wherein the different error correction codes comprise the first error correction code.
14. The data storage device of claim 13 , wherein:
each column of the bit array is associated with a different bit significance; and
columns associated with a lower bit significance are encoded using weaker error correction codes than columns associated with a higher bit significance.
15. The data storage device of claim 10 , wherein:
the memory comprises multiple types; and
the first portion of the memory and the second portion of the memory are of a first type of memory.
16. The data storage device of claim 15 , wherein:
file metadata associated with input files and inference results obtained based on the input files, are stored in a third portion of the memory.
17. The data storage device of claim 16 , wherein:
the input files are stored in a fourth portion of the memory; and
the fourth portion of the memory is of a second type of memory.
18. A non-transitory machine-readable medium having executable instructions to cause one or more processing devices to perform operations comprising:
organizing various types of data for a neural network into two or more hierarchies, such that data in different hierarchies is stored in different portions of memory, the memory comprising a plurality of dies;
storing weight data comprising a first and second subset of weights for the neural network;
the first subset of weights for the neural network is stored in a first portion of the memory, the neural network comprising a plurality of nodes and a plurality of connections between the plurality of nodes, wherein:
the first subset of weights comprises weights used by the neural network; and
a first error correction code is used for the first portion of the memory; and
the second subset of weights for the neural network is stored in a second portion of the memory, wherein:
more overprovisioning is utilized in the second portion of the memory than in the first portion of the memory;
the second subset of weights comprises weights that are prunable;
a second error correction code is used for the second portion of the memory; and
the second error correction code is weaker than the first error correction code,
wherein copies of the weight data are duplicated across the plurality of dies.