Enhanced memory device architecture for machine learning
Embodiments of an improved memory architecture for processing data inside of a device are described. In some embodiments, the device can store neural network layers, such as a systolic flow engine, in non-volatile memory and/or a separate first memory. A processor of a host system can delegate the execution of a neural network to the device. Advantageously, neural network processing in the device can be scalable, with the ability to process large amounts of data.
1. A device configured to perform neural network computations, the device comprising:
a first memory;
a second memory configured to store one or more layers of a neural network; and
means for:
storing data in at least one of the first memory or the second memory and retrieving data from at least one of the first memory or the second memory in response to at least one data transfer command received from a host system;
performing neural network computations in the second memory by applying one or more neural network layers to input data received from the host system; and
asynchronously storing a result of the neural network computations in the first memory for retrieval by the host system before completion of neural network computations for all neural network layers stored in the second memory.
2. The device of claim 1 , wherein the input data is stored in the first memory.
3. The device of claim 1 , further comprising means for:
performing neural network computations for a plurality of neural networks; and
using a result of neural network computations for a first neural network as input data for a successive neural network.
4. The device of claim 3 , further comprising means for reconfiguring the first neural network as the successive neural network before inputting the data into the successive network.
5. The device of claim 1 , wherein the means comprise a sole controller of the device.
6. The device of claim 1 , further comprising means for at least one of polling a state of memory pages in the second memory to determine the result or issuing an interrupt to determine the result.
7. The device of claim 6 , wherein the polling comprises periodic polling of the state of memory pages.
8. The device of claim 1 , further comprising means for receiving a request to initiate neural network computations, the request comprising neural network configuration parameters and input data for neural network computations.
9. The device of claim 8 , wherein the request to initiate neural network computations comprises a type of data processing, and wherein the device further comprises means for identifying neural network configuration parameters based on the type of data processing.
10. The device of claim 1 , wherein the first memory is a non-volatile memory.
11. A device configured to perform neural network computations, the device comprising:
a first memory;
a second memory configured to store one or more layers of a neural network; and
means for:
storing data in at least one of the first memory or the second memory and retrieving data from at least one of the first memory or the second memory in response to at least one data transfer command received from a host system;
performing neural network computations in the second memory by applying one or more neural network layers to input data received from the host system; and
synchronously storing a result of the neural network computations in the first memory for retrieval by the host system following completion of neural network computations for all neural network layers stored in the second memory.
12. The device of claim 11 , further comprising means for:
setting a locked state of the data before inputting the data into the neural network; and
setting an unlocked state of the data after making the output of the neural network available, wherein the locked state prevents changing the data.
13. The device of claim 11 , further comprising means for configuring the neural network based on at least one of a number of nodes or a type of activation function.
14. The device of claim 11 , further comprising means for inputting the data into the neural network by initiating back propagation on the neural network, wherein output of the neural network includes an adjusted weighting for one or more nodes of the neural network.
15. The device of claim 11 , wherein the first memory is a non-volatile memory.
16. A method of performing neural network computations in a device, the method comprising:
receiving at least one data transfer command from a host system;
storing data in at least one of a first memory or a second memory of the device, and retrieving data from at least one of the first memory or the second memory in response to the at least one data transfer command;
performing neural network computations for a plurality of neural networks in the second memory by applying neural network layers to input data received from the host system, wherein a first result of neural network computations for a first neural network is used as input data for a successive neural network; and
storing a result of the neural network computations in the first memory for retrieval by the host system.
17. The method of claim 16 , further comprising:
receiving a request to initiate neural network computations comprising a type of data processing; and
identifying neural network configuration parameters based on the type of data processing.
18. The method of claim 16 , wherein the plurality of neural networks is not directly accessible by a processor of the host system.
19. The method of claim 16 , further comprising:
receiving neural network configuration parameters and input data for the neural network computations; and
defining one or more neural network layers based on the neural network configuration parameters.
20. The method of claim 16 , further comprising:
receiving a request to perform a data processing function comprising a type of data processing;
identifying neural network configuration parameters based on the type of data processing; and
defining one or more neural network layers based on the neural network configuration parameters.