Multi-input multi-output time encoding and decoding machines
View Patent ↗Methods and systems for encoding and decoding signals using a Multi-input Multi-output Time Encoding Machine (TEM) and Time Decoding Machine are disclosed herein.
1. A multiple-input, multiple-output encoder comprising:
a filtering kernel comprising a plurality (N) of sets of filters, each of the N sets of filters having a plurality (M) of filter elements; and
at least one time encoding machine (TEM), each of the M filter elements of the N sets of filter elements being additively coupled to a corresponding input of the at least one TEM.
2. The encoder of claim 1 wherein the at least one TEM comprises an integrate-and-fire neuron.
3. The encoder of claim 1 wherein the at least one TEM comprises a neuron having multiplicative coupling.
4. The encoder of claim 1 wherein the at least one TEM comprises an asynchronous sigma/delta modulator.
5. The encoder of claim 1 further comprising a plurality (N) of adders, each adder coupled to a corresponding one of the N sets of filters and one of a plurality (N) of bias values.
6. The encoder of claim 1 wherein the at least one TEM comprises an irregular sampler.
7. The encoder of claim 1 wherein the at least one TEM generates a spike sequence.
8. The encoder of claim 1 wherein the at least one TEM comprises at least two ON-OFF neurons, each neuron being interconnected and having a feedback mechanism.
9. A decoder to decode a signal encoded by a multiple-input, multiple-output TEM encoder comprising:
at least one Time Decoding Machine (TDM); and
a filtering kernel comprising a plurality (N) of sets of filters, each of the N sets of filters having a plurality (M) of filter elements, a corresponding output of the at least one TDM being coupled to a corresponding one of the N sets of filters, each of the M filter elements of each set of filters being additively coupled to a corresponding M filter element of each other set of the N sets of filters.
10. The decoder of claim 9 , wherein the filtering kernel has an overall response selected to at least substantially invert an encoding process of a TEM encoder used to encode the TEM-encoded signal.
11. The decoder of claim 9 , wherein the TEM-encoded signal is irregularly sampled.
12. An encoder to encode a video stream signal comprising:
a filter to receive the video stream signal, the filter having a plurality (N) of filter elements; and
a Time Encoding Machine (TEM) to generate a plurality (N) of TEM encoded spatiotemporal field signals in response to a spatiotemporal field signal received from a corresponding one of the N filter elements.
13. The encoder of claim 12 , wherein the spatiotemporal field signals are described by an equation:
v j ( t )=∫ −∞ +∞ (∫∫ X D j ( x,y,s ) I ( x,y,t−s ) dxdy ) ds,
where D j (x,y,s) is a filter function, and I(x,y,t) represents the video stream signal.
14. The encoder of claim 12 wherein the N TEM-encoded spatiotemporal field signals are represented by a sampling function:
ψ k j (x,y,t)=D(x,y,−t)*g(t−s k j ), for k spike times, for each (x, y) in a bounded spatial set, where j corresponds to each of the N TEM-encoded spatiotemporal field signals, and where g(t)=sin(Ωt)/πt.
15. A decoder to decode a TEM-encoded video stream signal comprising:
a Time Decoding Machine (TDM) to receive a plurality (N) of TEM-encoded spatiotemporal field signals, and for each TEM-encoded spatiotemporal field signal generate a TDM-decoded spatiotemporal field signal; and
an adder to combine each of said TDM-decoded spatiotemporal field signals to recover the video stream signal.
16. A system to alter a video stream signal comprising:
a plurality (N) of TEM-filters;
a plurality (N) of reconstruction filters;
a switching matrix, operatively coupled to the plurality (N) of TEM-filters and the plurality (N) of reconstruction filters, to map a plurality (N) of TEM-encoded spatiotemporal field signals received from the N TEM-filters to the plurality (N) of reconstruction filters in a video stream signal TDM.
17. The system of claim 16 , adapted to rotate the video stream signal, wherein the switching matrix maps each of the N TEM-encoded spatiotemporal field signals from a corresponding TEM-filter ([x,y], α, θ) to a corresponding reconstruction filter ([x,y],α,θ+lθ 0 ), where lθ 0 represents a desired value of rotation.
18. The system of claim 16 , adapted to zoom the video stream signal, wherein the switching matrix maps each of the N TEM-encoded spatiotemporal field signals from a corresponding TEM-filter ([x,y], α, θ) to a corresponding reconstruction filter ([x,y], α 0 m α, θ), where α 0 m represents a desired value of zoom.
19. The system of claim 16 , adapted to translate the video stream signal by a value [nb 0 , kb 0 ], wherein the switching matrix maps each of the N TEM-encoded spatiotemporal field signals from a corresponding TEM-filter ([x,y], α, θ) to a corresponding reconstruction filter at ([x+nb 0 , y+kb 0 ], α, θ).
20. The system of claim 16 , adapted to zoom the video stream signal by a value α 0 m and translate said video stream signal by a value [nb 0 , kb 0 ], wherein the switching matrix maps each of the N TEM-encoded spatiotemporal field signals from a corresponding TEM-filter ([x,y], α, θ) to a corresponding reconstruction filter at ([x+α 0 m nb 0 , y+α 0 m kb 0 ], α 0 m α, θ).