Multi-input multi-output time encoding and decoding machines
View Patent ↗Methods and systems for encoding and decoding signals using a Multi-input Multi-output Time Encoding Machine (TEM) and Time Decoding Machine are disclosed herein.
1. A method for encoding a plurality (M) of components of a signal comprising:
filtering each of said M components into a plurality (N) of filtered signals; and
encoding each of said N filtered-signals using at least one Time Encoding Machine (TEM) to generate a plurality (N) of TEM-encoded filtered signals.
2. The method of claim 1 wherein said at least one TEM is an integrate-and-fire neuron.
3. The method of claim 1 wherein said at least one TEM has multiplicative coupling.
4. The method of claim 1 wherein said at least one TEM is an asynchronous sigma/delta modulator.
5. The method of claim 1 further comprising adding to each of said N filtered-signals one of a plurality (N) of bias values.
6. The method of claim 1 wherein said M signals are irregularly sampled.
7. The method of claim 1 wherein each of said N filtered signals is represented by the equation
v j =( h j ) T *u,j,
where H j =[h j1 , h j2 , . . . h jM ] T is a filtering vector corresponding to one of said N filtered signals represented by j.
8. The method of claim 1 wherein said N TEM-encoded filtered signals are represented by q k j where q k j =κ j δ j −b j (t k+1 j −t k j ), for all times represented by a value kε , and each of said N filtered signals represented by a value j, j=1, 2, . . . N, where κ j is an integration constant, δ j is a threshold value, and b j is a bias value for each of said N filtered signals.
9. A method for decoding a TEM-encoded signal comprising:
receiving a plurality (N) of TEM-encoded filtered signals;
decoding said N TEM-encoded filtered signals using at least one Time Decoding Machine (TDM) to generate a plurality (N) of TDM-decoded signal components; and
filtering each of said N TDM-decoded signal components into a plurality (M) of output signals components.
10. The method of claim 9 , wherein one of said M output signal components, represented by the i-th component of the vector valued signal u, |u i (t)|≦c i , can be recovered by solving for
u
i
(
t
)
=
∑
j
=
1
N
∑
k
∈
c
k
j
ψ
k
ji
(
t
)
,
where
ψ k ji (t)=({tilde over (h)} ji *g)(t−s k j ), for all i, i=1, 2, . . . M,
s k j =(t k+1 j +t k j )/2,
{tilde over (h)} ji is the involution of h ji ,
h ji is represented in a TEM-filterbank
h
(
t
)
=
[
h
11
(
t
)
h
12
(
t
)
⋯
h
1
M
(
t
)
h
21
(
t
)
h
22
(
t
)
⋯
h
2
M
(
t
)
⋮
⋮
⋱
⋮
h
N
1
(
t
)
h
N
2
(
t
)
⋯
h
NM
(
t
)
]
,
and
[c j ] k =c k 1 j=1, 2, . . . N, where c=[c 1 , c 2 , . . . , c N ] T , c=G + q, where q=[q 1 , q 2 , . . . , q N ] T and [q j ] k =q k j and
[
G
ij
]
kl
=
∑
m
=
1
M
∫
i
k
k
j
+
1
h
im
*
h
~
jm
*
g
(
t
-
s
l
j
)
ⅆ
s
.
11. The method of claim 9 , wherein said M TEM-encoded signals are irregularly sampled.
12. A method of encoding a video stream signal comprising:
filtering said video stream signal into a plurality (N) of spatiotemporal field signals; and
encoding each of said N spatiotemporal field signals with a Time Encoding Machine to generate a plurality (N) of TEM-encoded spatiotemporal field signals.
13. The method of claim 12 wherein said spatiotemporal field signals can be described by an equation:
v
j
(
t
)
=
∫
-
∞
+
∞
(
∫
∫
X
D
j
(
x
,
y
,
s
)
I
(
x
,
y
,
t
-
s
)
ⅆ
x
ⅆ
y
)
ⅆ
s
,
where D i (x,y,s) is a filter function, and I(x, y, t) represents said input video stream.
14. The method of claim 12 wherein said N TEM-encoded spatiotemporal field signals are represented by a sampling function:
ψ k j (x,y,t)=D(x,y,−t)*g(t−s k j ), for k spike times, for each (x, y) in a bounded spatial set, where j corresponds to each of said N TEM-encoded spatiotemporal field signals, and where g(t)=sin(Ωt)/πt.
15. A method of decoding a TEM-encoded video stream signal comprising:
receiving a plurality (N) of TEM-encoded spatiotemporal field signals;
decoding each of said N TEM-encoded spatiotemporal field signals using a Time Decoding Machine (TDM) to generate a TDM-decoded spatiotemporal field signal; and
combining each of said TDM-decoded spatiotemporal field signals to recover said video stream signal.
16. The method of claim 15 wherein said decoding and said combining are achieved by applying an equation:
I
(
x
,
y
,
t
)
=
∑
j
=
1
N
∑
k
∈
c
k
j
ψ
k
j
(
x
,
y
,
t
)
where ψ k j (x,y,t)=D(x,y,t)*g(t−s k j ), for k spike times, for each (x, y) in a bounded spatial set, where j corresponds to each of said N TEM-encoded spatiotemporal field signals, and where g(t)=sin(Ωt)/πt, and
where [c j ] k =c k j and c=[c 1 , c 2 , . . . c 1 ] T , c=G + q, where T denotes a transpose, q=[q 1 , q 2 , . . . q N ] T , [q j ] k j and G + denotes a pseudoinverse, a matrix G is represented by
G
=
[
G
11
G
12
⋯
G
1
N
G
21
G
22
⋯
G
2
N
⋮
⋮
⋱
⋮
G
N
1
G
N
2
⋯
G
NN
]
,
and
[
G
ij
]
kl
=
<
D
i
(
x
,
y
,
·
)
*
g
(
·
-
t
k
i
)
,
D
j
(
x
,
y
,
·
)
*
g
(
·
-
t
l
j
)
.
17. A method of altering a video stream signal comprising:
receiving a plurality (N) of TEM-encoded spatiotemporal field signals from a plurality (N) of TEM-filters; and
applying a switching matrix to map said N TEM-encoded spatiotemporal field signals to a plurality (N) of reconstruction filters in a video stream signal TDM.
18. The method of claim 17 for rotating said video stream signal wherein said switching matrix maps each of said N TEM-encoded spatiotemporal field signals from a TEM-filter ([x,y], α, θ) to a reconstruction filter ([x,y], α, θ+lθ 0 ), where lθ 0 represents a desired value of rotation.
19. The method of claim 17 for zooming said video stream signal wherein said switching matrix maps each of said N TEM-encoded spatiotemporal field signals from a TEM-filter ([x,y], α, θ) to a reconstruction filter ([x,y], α 0 m α, θ), where α 0 m represents a desired value of zoom.
20. The method of claim 17 for translating said video stream signal by a value [nb 0 , kb 0 ] wherein said switching matrix maps each of said N TEM-encoded spatiotemporal field signals from a TEM-filter ([x,y], α, θ) to a reconstruction filter at ([x+nb 0 , y+kb 0 ], α, θ).
21. The method of claim 17 for zooming said video stream signal by a value α 0 m and translating said video stream signal by a value [nb 0 , kb 0 ] wherein said switching matrix maps each of said N TEM-encoded spatiotemporal field signals from a TEM-filter ([x,y], α, θ) to a reconstruction filter at ([x+α 0 m nb 0 , y+α 0 m kb 0 ], α 0 m α, θ).
22. A method of encoding a video signal comprising:
inputting said video signal into a first and second time encoding machine (TEM), said first TEM comprising a first TEM-input and a first TEM-output, said second TEM comprising a second TEM-input and a second TEM-output;
wherein said first TEM-output is connected to said first TEM-input and said second TEM-input to provide negative feedback and said second TEM-output is connected to said first TEM-input and said second TEM-input to provide positive feedback.
23. The method of claim 22 further comprising:
outputting a first set of trigger values from said first TEM according to an equation
u
(
t
k
1
)
=
+
δ
1
+
∑
l
<
k
h
11
(
t
k
1
-
t
l
1
)
-
∑
l
h
21
(
t
k
1
-
t
l
2
)
1
{
t
l
2
<
t
k
1
}
=
q
k
1
;
and
outputting a second set of trigger values from said second TEM according to an equation
u
(
t
k
2
)
=
-
δ
2
+
∑
l
<
k
h
22
(
t
k
2
-
t
l
2
)
-
∑
l
h
12
(
t
k
2
-
t
l
1
)
1
{
t
l
1
<
t
k
2
}
=
q
k
2
.
24. The method of claim 22 further comprising:
outputting a first set of trigger values from said first TEM according to an equation
∫
1
k
k
1
+
1
u
(
s
)
ⅆ
s
=
κ
1
δ
1
-
b
1
(
t
k
+
1
1
-
t
k
1
)
+
∑
l
<
k
∫
1
k
k
1
+
1
h
11
(
s
-
t
l
1
)
-
∑
l
∫
1
k
k
1
+
1
h
21
(
s
-
t
l
2
)
ⅆ
s
1
{
t
l
2
<
t
k
1
}
;
and
outputting a second set of trigger values from said second TEM according to an equation
∫
2
k
k
2
+
1
u
(
s
)
ⅆ
s
=
κ
2
δ
2
-
b
2
(
t
k
+
1
1
-
t
k
1
)
+
∑
l
<
k
∫
2
k
k
2
+
1
h
22
(
s
-
t
l
2
)
-
∑
l
∫
2
k
k
2
+
1
h
12
(
s
-
t
l
1
)
ⅆ
s
1
{
t
l
1
<
t
k
2
}
.
25. A method of decoding a video signal comprising:
receiving first and second TEM-encoded signals; and
applying an equation
u
(
t
)
=
∑
k
∈
c
k
1
ψ
k
1
(
t
)
+
∑
k
∈
c
k
2
ψ
k
2
(
t
)
,
where ψ k j (t)=g(t−t k j ), for j=1, 2, g(t)=sin(Ωt)/πt, tε , c=[c 1 ; c 2 ] and [c j ] k =c k j , and a vector of coefficients c can be computed as c=G + q, where q=[q 1 ; q 2 ] and [q j ] k =q k j and
G
=
[
G
11
G
12
G
21
G
22
]
,
[
G
ij
]
kl
=
<
x
k
i
,
ψ
l
j
>
,
for all j=1, 2, and k, lε .