Analog neural memory system for deep learning neural network comprising multiple vector-by-matrix multiplication arrays and shared components
Numerous embodiments are disclosed for an analog neuromorphic memory system for use in a deep learning neural network. In one embodiment, the analog neuromorphic memory system comprises a plurality of vector-by-matrix multiplication systems, each vector-by-matrix multiplication system comprising an array of memory cells, a low voltage row decoder, a high voltage row decoder, and a low voltage column decoder; a plurality of output blocks, each output block providing an output in response to at least one of the plurality of vector-by-matrix multiplication systems; and a shared verify block configured to concurrently perform a verify operation after a program operation on two or more of the plurality of vector-by-matrix systems.
1. An analog neuromorphic memory system, comprising:
a plurality of vector-by-matrix multiplication systems, each vector-by-matrix multiplication system comprising:
an array of memory cells;
a low voltage row decoder;
a high voltage row decoder; and
a low voltage column decoder,
a plurality of output blocks, each output block providing an output in response to at least one of the plurality of vector-by-matrix multiplication systems; and
a shared verify block configured to perform a verify operation after a program operation concurrently on two or more of the plurality of vector-by-matrix systems.
2. The system of claim 1 , further comprising a sensing block for sensing ‘0’ values in memory cells, wherein the sensing block can be coupled to one or more of the plurality of vector-by-matrix multiplication systems.
3. The system of claim 1 , further comprising a sensing block for sensing ‘1’ values in memory cells, wherein the sensing block can be coupled to one or more of the plurality of vector-by-matrix multiplication systems.
4. The system of claim 1 , further comprising:
a shared sensing block configured to concurrently perform a read operation of memory cells in two or more of the plurality of vector-by-matrix systems.
5. The system of claim 1 , further comprising:
a shared trim block configured to adjust one or more components in any of the plurality of vector-by-matrix systems.
6. The system of claim 5 , wherein the one or more components comprise an adjustable resistor.
7. The system of claim 5 , wherein the one or more components comprise an adjustable transistor.
8. The system of claim 5 , wherein the one or more components comprise an adjustable current source.
9. The system of claim 1 , wherein the shared verify block is able to concurrently perform a read operation on memory cells in all of the vector-by-matrix multiplication systems.
10. The system of claim 1 , wherein the memory cells are split-gate flash memory cells.
11. The system of claim 1 , wherein each vector-by-matrix multiplication system is a cell in a long short term memory system.
12. The system of claim 1 , wherein each vector-by-matrix multiplication system is a cell in a gated recurrent unit memory system.
13. The system of claim 1 , wherein each of the plurality of output blocks comprises a summer and activation block.
14. The system of claim 13 , wherein each summer and activation block is configured to perform a summing and activation function for at least one of the plurality of vector-by-matrix multiplication systems.