Enhanced storage device memory architecture for machine learning
View Patent ↗Embodiments of storage device architecture for processing data using machine learning are disclosed. In some embodiments, the storage device includes a separate I/O core and a neural network core. The storage device can create a copy of data streams in which the data is stored, and the neural network core can process the copy of the data streams in a neural network while the I/O core can perform read or write functions on the data streams.
1. A data storage device configured to perform neural network computations, the device comprising:
a non-volatile memory comprising a first memory region configured to store data provided by a host system and a second memory region configured to store data related to neural network computations; and
at least one controller configured to:
store data in the first memory region and retrieve data from the first memory region in response to at least one data transfer command received from the host system;
receive, from the host system, a request to perform an operation on data stored in the first memory region;
in response to determining that the data stored in the first memory region is in a locked state, store the request in a journal to defer performance of the operation on the data stored in the first memory region;
copy the first memory region to the second memory region;
unlock the first memory region; and
perform neural network computations on copied data in the second memory region while performing the operation on the data stored in the first memory region using the request stored in the journal.
2. The device of claim 1 , wherein the second memory region is configured to store a plurality of memory streams, each stream including a contiguous set of physical memory storage units of the non-volatile memory, and wherein the at least one controller is further configured to perform neural network computations on the plurality of memory streams.
3. The device of claim 2 , wherein the at least one controller is further configured to identify each memory stream of the plurality of memory streams by a common identifier.
4. The device of claim 2 , wherein the plurality of memory streams comprises a first memory stream and a second memory stream.
5. The device of claim 2 , wherein the at least one controller is further configured to store input data for neural network computations in at least one memory stream of the plurality of memory stream.
6. The device of claim 2 , wherein the plurality of memory streams comprises at least one input memory stream and at least one output memory stream, and wherein the at least one controller is further configured to perform neural network computations on data stored in the at least one input memory stream and store a result of the neural network computations in the at least one output memory stream.
7. The device of claim 6 , wherein the at least one controller is further configured to receive from the at least one output memory stream the result of the neural network computations and provide the result to the host system.
8. The device of claim 2 , wherein the at least one controller comprises a plurality of processor cores configured to process a plurality of memory streams substantially concurrently.
9. The device of claim 1 , wherein the at least one controller includes an I/O core and a neural network core.
10. A method for performing neural network computations within a data storage device, the method comprising, by at least one controller of the data storage device:
receiving, from a host system, a first request to perform processing of data stored in a memory region of a non-volatile memory of the data storage device;
locking the memory region of the non-volatile memory;
receiving, from the host system, a second request to perform an operation on data stored in the memory region of the non-volatile memory;
in response to determining that the memory region is in a locked state, storing the second request in a journal to defer performance of the operation in the memory region;
copying of the memory region of the non-volatile memory;
unlocking the memory region of the non-volatile memory;
initiating processing of the data by applying a neural network on copied data of the copied memory region; and
in the unlocked memory region, performing the operation for the second request on the data stored in the memory region using the second request stored in the journal while the processing of the copied data of the copied memory region is performed by the neural network.
11. The method of claim 10 , wherein the neural network includes a systolic flow engine.
12. The method of claim 10 , wherein neural network parameters are stored in the non-volatile memory, and the processing of the data via the neural network occurs within the data storage device.
13. The method of claim 10 , wherein the operation includes a write operation.
14. The method of claim 10 , wherein the method further comprises, by the at least one controller:
in response to determining that the processing of the data via the neural network has been completed, deleting the copied memory region.
15. A data storage device configured to perform neural network computations, the device comprising:
a non-volatile memory comprising a first memory region configured to store data provided by a host system and a second memory region configured to store data related to neural network computations;
a first controller configured to:
receive, from the host system, a first request to perform analysis of data stored in the first memory region;
set a locked state for the data of the first memory region; and
copy the data stored in the first memory region to the second memory region; and
a second controller configured to:
receive, from the host system, a second request to perform an operation on data stored in the first memory region;
in response to determining that the data stored in the first memory region is in the locked state, store the second request in a journal to defer performance of the operation on the data stored in the first memory region;
determine that the data stored in the first memory region is in an unlocked state; and
in response to determining that the data stored in the first memory region is in the unlocked state, perform the operation for the second request on the data stored in the unlocked first memory region using the second request stored in the journal while processing of copied data in the second memory region is performed by the neural network.
16. The device of claim 15 , wherein the journal is configured to prevent writing to the first memory region.
17. The storage device of claim 15 , wherein the first controller is further configured to:
receive an additional request to perform analysis of data stored in the first memory region while the data stored in the first memory region is in the locked state; and
copy the data stored in the first memory region into a third memory region without waiting for the data stored in the first memory region to be in an unlocked state.
18. The device of claim 15 , wherein the first controller is further configured to retrieve from an output memory stream the result of the neural network computations and provide the result to the host system, the output memory stream comprising a contiguous set of physical memory storage units of the second memory region of the non-volatile memory.
19. The device of claim 15 , wherein the first controller comprises a plurality of processing cores configured to process a plurality of memory streams substantially concurrently.