Configurable analog neural memory system for deep learning neural network
Numerous embodiments are disclosed for a configurable hardware system for use in an analog neural memory system for a deep learning neural network. The components within the configurable hardware system that are configurable can include vector-by-matrix multiplication arrays, summer circuits, activation circuits, inputs, reference devices, neurons, and testing circuits. These devices can be configured to provide various layers or vector-by-matrix multiplication arrays of various sizes, such that the same hardware can be used in analog neural memory systems with different requirements.
1. A configurable vector-by-matrix multiplication system, comprising:
an array of memory cells arranged into rows and columns;
an output block coupled to the array for generating a vector of output voltages in response to current received from a plurality of memory cells in the array during a vector matrix multiplier operation;
an activation block coupled to the array for generating a vector of input currents in response to a vector of input voltages and providing the vector of input currents to a plurality of memory cells in the array during a vector matrix multiplier operation, wherein during a first cycle, a first sub-array is generated in the array and the output block is coupled to the first sub-array and the activation block is coupled to the first sub-array, and wherein during a second cycle, a second sub-array is generated within the array and the output block is coupled to the second sub-array and the activation block is coupled to the second sub-array, the first sub-array and second sub-array consisting of different memory cells in the array; and
routing circuitry for routing a vector of output voltages from the output block in response to current received from the first sub-array to the activation block for vector matrix multiplier operation of the second sub-array.
2. The system of claim 1 , wherein the routing circuitry comprises one or more multiplexors.
3. The system of claim 1 , further comprising a controller for generating the first sub-array and the second sub-array.
4. The system of claim 1 , further comprising control logic for generating the first sub-array and the second sub-array.
5. The system of claim 1 , wherein the memory cells are split-gate flash memory cells.
6. The system of claim 1 , wherein the output block is a current summer block.
7. The system of claim 1 , wherein the system provides a capability of configuring an output width of a neuron within the system.
8. The system of claim 1 , wherein the system provides a capability of configuring an input width of a neuron within the system.
9. The system of claim 1 , wherein the output block outputs digital bits.
10. The system of claim 1 , further comprising one or more of a current sample-and-hold circuit or a voltage sample-and-hold circuit.
11. The system of claim 1 , wherein the vector-by-matrix multiplication system is part of a long short term memory cell.
12. The system of claim 1 , wherein the vector-by-matrix multiplication system is part of a gated recurrent unit cell.
13. A configurable vector-by-matrix multiplication system, comprising:
an array of memory cells arranged into rows and columns;
an output block coupled to the array for generating a vector of outputs in response to current received from a plurality of memory cells in the array during a vector matrix multiplier operation, wherein during a first cycle, a first sub-array is generated in the array and the output block is coupled to the first sub-array and the activation block is coupled to the first sub-array, and wherein during a second cycle, a second sub-array is generated within the array and the output block is coupled to the second sub-array and the activation block is coupled to the second sub-array, the first sub-array and second sub-array consisting of different memory cells in the array; and
routing circuitry for routing a vector of output voltages from the output block in response to current received from the first sub-array to the activation block for vector matrix multiplier operation of the second sub-array.
14. The system of claim 13 , wherein the routing circuitry comprises one or more multiplexors.
15. The system of claim 13 , further comprising a controller for generating the first sub-array and the second sub-array.
16. The system of claim 13 , further comprising control logic for generating the first sub-array and the second sub-array.
17. The system of claim 13 , wherein the memory cells are split-gate flash memory cells.
18. The system of claim 13 , wherein the output block is a current summer block.
19. The system of claim 13 , wherein the system provides a capability of configuring an output width of a neuron within the system.
20. The system of claim 13 , wherein the system provides a capability of configuring an input width of a neuron within the system.
21. The system of claim 13 , wherein the output block outputs digital bits.
22. The system of claim 13 , wherein the output block outputs analog levels.
23. The system of claim 13 , further comprising one or more of a current sample-and-hold circuit and a voltage sample-and-hold circuit.
24. The system of claim 13 , wherein the output block comprises an analog-to-digital conversion block.
25. The system of claim 13 , further comprising an input block.
26. The system of claim 25 , wherein the input block comprises an activation block.
27. The system of claim 25 , wherein the input block comprises a digital-to-analog conversion block.
28. The system of claim 13 , wherein the vector-by-matrix multiplication system is part of a long short term memory cell.
29. The system of claim 13 , wherein the vector-by-matrix multiplication system is part of a gated recurrent unit cell.
30. The system of claim 13 , wherein a weight is stored in a differential cell.
31. The system of claim 13 , wherein a weight is stored in two blend memory cells.
32. A configurable vector-by-matrix multiplication system, comprising:
an array of memory cells arranged into rows and columns;
an input block coupled to the array for generating a vector of inputs in response to input data to a plurality of memory cells in the array during a vector matrix multiplier operation, wherein during a first cycle, a first sub-array is generated in the array and the input block is coupled to the first sub-array, and wherein during a second cycle, a second sub-array is generated within the array and the input block is coupled to the second sub-array, the first sub-array and second sub-array consisting of different memory cells in the array; and
routing circuitry for routing a vector of input vector from the input block in response to input data coupled to the first sub-array for vector matrix multiplier operation.
33. The system of claim 32 , wherein the routing circuitry comprises one or more multiplexors.
34. The system of claim 32 , further comprising a controller for generating the first sub-array and the second sub-array.
35. The system of claim 32 , further comprising control logic for generating the first sub-array and the second sub-array.
36. The system of claim 32 , wherein the memory cells are split-gate flash memory cells.
37. The system of claim 32 , further comprising an output block generating an output vector.
38. The system of claim 37 , wherein the output block is a current summer block.
39. The system of claim 32 , wherein the system provides a capability of configuring an output width of a neuron within the system.
40. The system of claim 32 , wherein the system provides a capability of configuring an input width of a neuron within the system.
41. The system of claim 32 , wherein the output block outputs digital bits.
42. The system of claim 32 , wherein the output block outputs analog levels.
43. The system of claim 32 , further comprising one or more of a current sample-and-hold circuit and a voltage sample-and-hold circuit.
44. The system of claim 32 , wherein the output block comprises an analog-to-digital conversion block.
45. The system of claim 33 , wherein the output block comprises an analog-to-digital conversion block.
46. The system of claim 32 , wherein the input block comprises an activation block.
47. The system of claim 32 , wherein the input block comprises a digital-to-analog conversion block.
48. The system of claim 32 , wherein the input block is coupled to a source gate of each memory cell in the array of memory cells.
49. The system of claim 32 , wherein the input block is coupled to a word line gate of each memory cell in the array of memory cells.
50. The system of claim 32 , wherein the vector-by-matrix multiplication system is part of a long short term memory cell.
51. The system of claim 32 , wherein the vector-by-matrix multiplication system is part of a gated recurrent unit cell.
52. The system of claim 32 , wherein a weight is stored in a differential cell.
53. The system of claim 32 , wherein a weight is stored in two blend memory cells.
54. An analog neuro memory system, comprising:
a plurality of vector-by-matrix multiplication sub-systems, each vector-by-matrix sub-system comprising:
an array of memory cells arranged into rows and columns;
an output block coupled to the array for generating a vector of output voltages in response to current received from a plurality of memory cells in the array during a vector matrix multiplier operation; and
an activation block coupled to the array for generating a vector of input currents in response to a vector of input voltages and providing the vector of input currents to a plurality of memory cells in the array during a vector matrix multiplier operation; and
routing circuitry for routing a vector of output voltages from an output block coupled to a first sub-array contained within a first array in one of the plurality of vector-by-matrix multiplication sub-systems to an activation block coupled to a second sub-array contained within a second array in another of the plurality of vector-by-matrix multiplication sub-systems.
55. The system of claim 54 , wherein the routing circuitry comprises one or more multiplexors.
56. The system of claim 54 , further comprising a controller for generating the first sub-array and the second sub-array.
57. The system of claim 54 , further comprising control logic for generating the first sub-array and the second sub-array.
58. The system of claim 54 , wherein some or all of the vector-by-matrix multiplication sub-systems form a long short term memory cell.
59. The system of claim 54 , wherein some or all of the vector-by-matrix multiplication sub-systems form a gated recurrent unit cell.
60. The system of claim 54 , wherein the memory cells are split-gate flash memory cells.
61. The system of claim 54 , wherein the output block is a current summer block.
62. The system of claim 54 , wherein the system provides a capability of configuring an output width of a neuron within the system.
63. The system of claim 54 , wherein the system provides a capability of configuring an input width of a neuron within the system.
64. The system of claim 54 , wherein the output block outputs digital bits.
65. The system of claim 54 , further comprising current sample-and-hold circuits or voltage sample-and-hold circuits.
66. The system of claim 54 , wherein the vector-by-matrix multiplication system is part of a long short term memory cell.
67. The system of claim 54 , wherein the vector-by-matrix multiplication system is part of a gated recurrent unit cell.