Analog system for computing sparse codes
A parallel dynamical system for computing sparse representations of data, i.e., where the data can be fully represented in terms of a small number of non-zero code elements, and for reconstructing compressively sensed images. The system is based on the principles of thresholding and local competition that solves a family of sparse approximation problems corresponding to various sparsity metrics. The system utilizes Locally Competitive Algorithms (LCAs), nodes in a population continually compete with neighboring units using (usually one-way) lateral inhibition to calculate coefficients representing an input in an over complete dictionary.
1. An analog system for sparsely approximating a signal comprising:
a matching system for calculating and outputting matching signals representative of how well-matched said signal is to a plurality of dictionary elements; and
a plurality of nodes, each node receiving one of said matching signals from said matching system, wherein each node comprises:
a source of an internal state signal; and
a thresholding element;
wherein said internal state signal in each node is calculated as a function of said matching signal received at said node and weighted outputs of all other nodes.
2. An analog system for sparsely approximating a signal according to claim 1 , wherein said matching system comprises a projection system for projecting a signal vector onto said plurality of dictionary elements.
3. An analog system for sparsely approximating a signal according to claim 1 wherein said source of an internal state signal comprises a low pass averaging system.
4. An analog system for sparsely approximating a signal according to claim 1 wherein each node further comprises a plurality of weighting elements for receiving an output of said thresholding element and providing a plurality of weighted outputs.
5. An analog system for sparsely approximating a signal according to claim 1 wherein each node further comprises a plurality of weighting elements, each weighting element receiving an output from another one of said plurality of nodes and providing a weighted output to said source of an internal state signal.
6. An analog system for sparsely approximating a signal according to claim 1 wherein said internal state signal is derived from said matching signal less a sum of weighted outputs from said other nodes.
7. An analog system for sparsely approximating a signal according to claim 1 wherein said signal comprises a video signal.
8. An analog system for sparsely approximating a signal according to claim 1 wherein said source of an activation signal comprises a voltage source.
9. An analog system for sparsely approximating a signal according to claim 3 wherein said low pass averaging system comprises a low pass averaging circuit.
10. An analog system for sparsely approximating a signal according to claim 8 wherein said low pass averaging circuit comprises a resistor and a capacitor.
11. An analog system for sparsely approximating a signal according to claim 1 wherein said source of an activation signal comprises a current source.
12. A parallel dynamical system for computing sparse representations of data comprising:
a projection system for projecting said data onto projection vectors; and
a plurality of nodes, each node receiving one of said projection vectors from said projection system, wherein each node comprises:
a source of electrical energy;
a low pass averaging circuit; and
a thresholding element;
wherein said source of electrical energy in each node comprises a projection vector received at said node less weighted outputs of all other nodes.
13. A parallel dynamical system for computing sparse representations of data according to claim 12 wherein each node further comprises a plurality of weighting elements, each weighting element receiving an output from another one of said plurality of nodes and providing said weighted output to said source of electrical energy.
14. A parallel dynamical system for computing sparse representations of data comprising:
a plurality of nodes, each node being active or inactive and each said node comprising:
a leaky integrator element, wherein inputs to said leaky integrator element cause an activation potential to charge up; and
a thresholding element for receiving said activation potential and for producing an output coefficient, said output coefficient being the result of an activation function applied to said activation potential and parameterized by a system threshold;
wherein active nodes inhibits other nodes with inhibition signals proportional to both level of activity of said active nodes and a similarity of receptive fields of said active nodes.