Enhanced memory device architecture for machine learning
Embodiments of an improved memory architecture by processing data inside of the memory device are described. In some embodiments, the memory device can store neural network layers, such as a systolic flow engine, in non-volatile memory and/or a separate DRAM memory. Central processing unit (CPU) of a host system can delegate the execution of a neural network to the memory device. Advantageously, neural network processing in the memory device can be scalable, with the ability to process large amounts of data.
1. A memory device configured to perform neural network computations, the device comprising:
a volatile memory;
a non-volatile memory configured to store one or more layers of a neural network; and
a controller configured to:
store data in at least one of the volatile memory or the non-volatile memory and retrieve data from at least one of the volatile memory or the non-volatile memory in response to at least one data transfer command received from a host system;
perform neural network computations in the non-volatile memory by applying one or more neural network layers to input data received from the host system;
store a result of the neural network computations in the volatile memory; and
provide the result of the neural network computations to the host system asynchronously before completion of neural network computations for all neural network layers stored in the non-volatile memory.
2. The device of claim 1 , wherein the input data is stored in the volatile memory.
3. The device of claim 1 , wherein the controller is further configured to perform neural network computations for a plurality of neural networks and use a result of neural network computations for a first neural network as input data for a successive neural network.
4. The device of claim 3 , wherein the controller is further configured to reconfigure the first neural network as the successive neural network before inputting the data into the successive network.
5. The device of claim 1 , wherein the controller is a sole controller of the memory device.
6. The device of claim 1 , wherein provision of the result asynchronously comprises at least one of polling a state of memory pages in the non-volatile memory or issuing an interrupt.
7. The device of claim 6 , wherein the polling comprises periodic polling of the state of memory pages.
8. The device of claim 1 , wherein the memory device is further configured to receive a request to initiate neural network computations, the request comprising neural network configuration parameters and input data for neural network computations.
9. The device of claim 8 , wherein the request to initiate neural network computations comprises a type of data processing, and wherein the controller is further configured to identify neural network configuration parameters based on the type of data processing.
10. A method of performing neural network computations in a memory device, the method comprising:
by a controller of the memory device:
storing data in at least one of a volatile memory of the memory device or a non-volatile memory of the memory device and retrieving data from at least one of the volatile memory or the non-volatile memory in response to at least one data transfer command received from a host system;
performing neural network computations in the non-volatile memory by applying one or more neural network layers of a neural network to input data received from the host system; and
storing a result of the neural network computations in the volatile memory for retrieval by the host system, wherein the result of the neural network computations is configured to be retrieved synchronously following completion of neural network computations for all neural network layers stored in the non-volatile memory.
11. The method of claim 10 , further comprising, by the controller, setting a locked state of the data before inputting the data into the neural network, and setting an unlocked state of the data after making the output of the neural network available, wherein the locked state prevents changing the data.
12. The method of claim 10 , further comprising configuring the neural network based on at least one of a number of nodes or a type of activation function.
13. The method of claim 10 , further comprising inputting the data into the neural network by initiating back propagation on the neural network, and wherein output of the neural network includes an adjusted weighting for one or more nodes of the neural network.
14. A data storage device comprising:
a first memory;
a second memory; and
a controller configured to:
store data in at least one of the first memory or the second memory and retrieve data from at least one of the first memory or the second memory in response to at least one data transfer command received from a host system;
perform neural network computations for a plurality of neural networks in the second memory by applying neural network layers to input data received from the host system and stored in the first memory, wherein a first result of neural network computations for a first neural network is used as input data for a successive neural network; and
store a result of the neural network computations in the first memory for retrieval by the host system.
15. The device of claim 14 , wherein the device is configured to receive a request to initiate neural network computations comprising a type of data processing, and wherein the controller is further configured to identify neural network configuration parameters based on the type of data processing.
16. The device of claim 14 , wherein the plurality of neural networks is not directly accessible by a processor of the host system.
17. The device of claim 14 , wherein the device is configured to receive neural network configuration parameters and input data for the neural network computations, and wherein the controller is further configured to define one or more neural network layers based on the neural network configuration parameters.
18. The device of claim 14 , wherein the device is configured to receive a request to perform a data processing function comprising a type of data processing, and wherein the controller is further configured to identify neural network configuration parameters based on the type of data processing and define one or more neural network layers based on the neural network configuration parameters.