Artificial intelligence memory architecture systems and methods
A system and method for providing a neural network model to an artificial intelligence (AI) field programmable gate array (FPGA) are provided. A flash memory stores a neural network model. A tunnel is created between a flash memory and a random access memory (RAM) over a multi-line serial peripheral interface (QSPI) interface. Using the tunnel, the RAM reads one or more layers of the neural network model from the flash memory and writes the one or more layers into pages in the RAM. The AI FPGA reads the one or more layers of the neural network model from the RAM over a wide input/output interface and executes the one or more layers.
1 . A system comprising:
a flash memory configured to store a neural network model and comprising a direct memory access (DMA) controller configured to create a tunnel over a first interface;
a random access memory (RAM) configured to read, over the first interface, one or more layers of the neural network model from the flash memory and write the one or more layers of the neural network model into the RAM; and
an artificial intelligence (AI) field programmable gate array (FPGA) and/or Application Specific Integrated Circuit (ASIC) configured to read the one or more layers of the neural network model from the RAM over a second interface and execute the one or more layers.
2 . The system of claim 1 , wherein the AI FPGA is further configured to issue a closed coupled memory (CCM) command to the flash memory over a third interface, wherein the CCM command initiates the DMA controller at the flash memory.
3 . The system of claim 2 , wherein the third interface is a multi-line serial peripheral interface (QSPI) interface.
4 . The system of claim 1 , wherein the first interface is a multi-line SPI interface.
5 . The system of claim 1 , wherein the one or more layers of the neural network model are written into a plurality of pages in the RAM.
6 . The system of claim 5 , wherein the AI FPGA is further configured to read a first layer of the neural network model stored in a first page from the plurality of pages over the second interface while the RAM is configured to write a second layer into a second page in the plurality of pages from the flash memory.
7 . The system of claim 1 , wherein the second interface is a wide input/output interface.
8 . The system of claim 1 , wherein the RAM is further configured to read and write the one or more layers of the neural network model over the first interface during boot-up of a computing device.
9 . The system of claim 1 , wherein the flash memory is next to the RAM and the RAM is next to the AI FPGA on a multi-chip module package on a chip.
10 . The system of claim 1 , wherein the flash memory further comprises a closed coupled memory (CCM) command decoder configured to interpret a CCM command from the AI FPGA, and wherein the DMA controller creates the tunnel in response to the CCM command.
11 . The system of claim 1 , wherein the DMA controller is configured to transfer the one or more layers of the neural network model from the flash memory to the RAM without involving a compute engine of the AI FPGA.
12 . A method comprising:
creating, using a direct memory access (DMA) controller in a flash memory, a tunnel over a first interface between the flash memory and a random access memory (RAM);
reading, using the tunnel over the first interface, from a flash memory, one or more layers of a neural network model;
writing, using the tunnel over the first interface, the one or more layers of the neural network model into the RAM;
reading, over a second interface, into an artificial intelligence (AI) field programmable gate array (FPGA) and/or Application Specific Integrated Circuit (ASIC) the one or more layers of the neural network model from the RAM; and
executing, using a compute engine of the AI FPGA, the one or more layers of the neural network model.
13 . The method of claim 12 , further comprising:
issuing, over a third interface, from the AI FPGA a closed coupled memory (CCM) command to the flash memory; and
initiating, using the CCM command, a direct memory access (DMA) controller at the flash memory.
14 . The method of claim 13 , wherein the first interface and/or the third interface is a multi-line SPI interface or a direct die-to-die interface.
15 . The method of claim 12 , wherein the one or more layers of the neural network model are written into a plurality of pages in the RAM.
16 . The method of claim 15 , further comprising:
reading, by the AI FPGA and over the second interface, a first layer of the neural network model stored in a first page from the plurality of pages while writing, by the RAM, a second layer into a second page in the plurality of pages from the flash memory.
17 . The method of claim 12 , wherein the second interface is a wide input/output interface or a direct die-to-die interface.
18 . The method of claim 12 , wherein the reading and the writing the one or more layers of the neural network model into the RAM occurs during initialization of a computing device.
19 . The method of claim 12 , wherein the flash memory is next to the RAM and the RAM is next to the AI FPGA on a multi-chip module package on a chip.
20 . A system comprising:
a flash memory configured to store a data structure and comprising a direct memory access (DMA) controller configured to create a tunnel over a first interface;
a random access memory (RAM) configured to load, using the tunnel over the first interface, a portion of the data structure from the flash memory into the RAM; and
an artificial intelligence (AI) field programmable gate array (FPGA) and/or Application Specific Integrated Circuit (ASIC) configured to read the portion of the data structure from the RAM over a second interface and execute the portion of the data structure.