Optimized neural network data organization
In some implementations, the present disclosure relates to a method. The method includes obtaining a set of weights for a neural network comprising a plurality of nodes and a plurality of connections between the plurality of nodes. The method also includes identifying a first subset of weights and a second subset of weights based on the set of weights. The first subset of weights comprises weights that used by the neural network. The second subset of weights comprises weights that are prunable. The method further includes storing the first subset of weights in a first portion of a memory. A first error correction code is used for the first portion of the memory. The method further includes storing the second subset of weights in a second portion of the memory. A second error correction code is used for the second portion of the memory. The second error correction code is weaker than the first error correction code.
1. A data storage device, comprising:
a memory comprising a plurality of dies; and
a controller coupled to the memory, the controller configured to:
organize data for a neural network across different portions of the memory; and
store weight data wherein the weight data comprises a first and second subset of weights for use by the neural network, wherein:
the first subset of weights is stored in a first portion of the memory, the first portion being configured to utilize a first error correction code;
the second subset of weights is stored in a second portion of the memory, the second portion being configured to utilize a second error correction code configured to be different than the first error correction code, and wherein the second subset of weights comprises prunable weights, that are pruned until a predetermined threshold associated with an accuracy of the neural network is exceeded.
2. The device of claim 1 , wherein the data is organized into two or more hierarchies across the different portions of the memory.
3. The device of claim 2 , wherein the first subset of weights stored in the first portion of the memory is associated with one of the two or more hierarchies.
4. The device of claim 3 , wherein the second subset of weights stored in the second portion of the memory is associated with another of the two or more hierarchies.
5. The device of claim 4 , wherein the weight data is stored across the plurality of dies.
6. The device of claim 1 , wherein the second error correction code is configured to be weaker than the first error correction code.
7. The device of claim 1 , wherein the weights within the weight data are ranked based on a plurality of evaluation metrics.
8. The device of claim 7 , wherein at least one of the plurality of evaluation metrics ranks the weights based on their effect on the accuracy of inferences or results generated by the neural network.
9. The device of claim 8 , wherein weights below a predetermined ranking are selected for pruning.
10. A method, comprising:
organizing data for a neural network into two or more hierarchies across different portions of a memory comprising a plurality of dies; and
storing and duplicating weight data across the plurality of dies wherein the weight data comprises a first and second subset of weights for use by the neural network, wherein:
the first subset comprises unprunable weights and is stored in a first portion of the memory configured to utilize a first error correction code; and
the second subset comprises prunable weights and is stored in a second portion of the memory configured to utilize a second error correction code that is different than the first error correction code.
11. The method of claim 10 , wherein the data is organized into two or more hierarchies across the different portions of the memory.
12. The device of claim 11 , wherein the weight data is further duplicated across the plurality of dies.
13. The device of claim 12 , wherein the second error correction code is configured to be weaker than the first error correction code.
14. The method of claim 10 , wherein the first and second subset of weights are stored in different types of memory.
15. The method of claim 14 , wherein the selection of the different type of memory to store the first and second subset of weights is based on the utilization of the weight data.
16. The method of claim 15 , wherein weight data that is accessed more often is stored within a memory type that has a lower latency compared to the other types of memory.
17. The method of claim 16 , wherein the weight data stored within a lower latency memory type is configured with a weaker error correction code.
18. A non-transitory machine-readable medium having executable instructions to cause one or more processing devices to perform operations comprising:
organizing data for a neural network across different portions of a memory comprising a plurality of memory structures; and
storing weight data across the plurality of memory structures wherein the weight data comprises a first and second subset of weights for use by the neural network, wherein:
the first subset comprises unprunable weights and is stored in a first portion of the memory configured to utilize a first error correction code, and
the second subset comprises prunable weights and is stored in a second portion of the memory configured to utilize a second error correction code that is different than the first error correction code, that are pruned until a predetermined threshold associated with an accuracy of the neural network is exceeded.