Tunable kernels for tracking
View Patent ↗A tunable representation for tracking that simultaneously encodes appearance and geometry in a manner that enables the use of mean-shift iterations for tracking is provided. The solution to the tracking problem is articulated into a method that encodes the spatial configuration of features along with their density and yet retains robustness to spatial deformations and feature density variations. The method of encoding of spatial configuration is provided using a set of kernels whose parameters can be optimized for a given class of objects off-line. The method enables the use of mean-shift iterations and runs in real-time. Better tracking results by the novel tracking method as compared to the original mean-shift tracker are demonstrated.
1. A method for creating a tunable representation of an object for mean-shift iterations tracking from a target region to a candidate region in consecutive image frames, comprising:
encoding by a processor of image data in the consecutive image frames representing a target region as a spatial configuration of an object using a plurality of kernels;
determining by the processor of an expression for a spatial configuration of the candidate region from the target region; and
optimizing by the processor of a bandwidth parameter of the plurality of kernels using prior knowledge of motion characteristics of the object.
2. The method as claimed in claim 1 , wherein:
a kernel is a block with a tunable bandwidth.
3. The method as claimed in claim 2 , further comprising:
encoding by the processor of a region by including bandwidth parameters of the plurality of kernels so that it penalizes an appearance of pixels at large distances from kernels where a similar appearance occurs; and
the processor using radially symmetric kernels.
4. The method as claimed in claim 2 , wherein a bandwidth of a block depends on a reach of motion.
5. The method as claimed in claim 1 , wherein a spatial density of a candidate region is expressed as:
p
^
u
(
y
)
=
C
p
∑
i
=
1
N
δ
[
b
(
x
i
)
-
u
]
∑
j
=
1
M
S
j
(
u
)
k
(
x
i
-
y
-
z
_
j
h
j
2
)
;
and wherein z j is the center of block j and h j is the bandwidth of block j.
6. The method as claimed in claim 5 , wherein a mean shift vector is expressed as:
y
(
t
+
1
)
=
∑
i
=
1
N
w
i
(
α
i
(
u
i
,
t
)
x
i
-
v
i
(
u
i
,
t
)
)
∑
i
=
1
N
w
i
α
i
(
u
i
,
t
)
;
and wherein
α
i
(
u
,
t
)
=
∑
j
=
1
M
1
h
j
2
S
j
(
u
)
C
ij
(
t
)
and
v
i
(
u
,
t
)
=
∑
j
=
1
M
1
h
j
2
S
j
(
u
)
C
ij
(
t
)
z
_
j
.
7. The method as claimed in claim 2 , wherein the optimized bandwidth of a kernel is estimated by using motion capture.
8. The method as claimed in claim 2 , wherein the object is articulated.
9. The method as claimed in claim 2 , wherein the bandwidth of the plurality of kernels is determined by the processor by maximizing a functional of a class-separation metric.
10. The method as claimed in claim 2 , wherein the bandwidth of the plurality of kernels is determined by the processor by maximizing a functional of a class-discrimination metric.
11. A system for mean-shift tracking an object in a target region to a candidate region in consecutive image frames, comprising:
a processor;
software operable on the processor for:
encoding a target region as a spatial configuration of an object using a plurality of kernels;
determining an expression for a spatial configuration of the candidate region from the target region; and
optimizing a bandwidth parameter of the plurality of kernels using prior knowledge of motion characteristics of the object.
12. The system as claimed in claim 11 , wherein:
a kernel is a block with a tunable bandwidth.
13. The system as claimed in claim 11 , further comprising:
encoding a region by including bandwidth parameters of the plurality of kernels so that it penalizes an appearance of pixels at large distances from kernels where a similar appearance occurs; and
using radially symmetric kernels.
14. The system as claimed in claim 12 , wherein a bandwidth of a block depends on a reach of motion of an object.
15. The system as claimed in claim 12 , wherein a spatial density of a candidate region is expressed as:
p
^
u
(
y
)
=
C
p
∑
i
=
1
N
δ
[
b
(
x
i
)
-
u
]
∑
j
=
1
M
S
j
(
u
)
k
(
x
i
-
y
-
z
_
j
h
j
2
)
;
and wherein z j is the center of a block j and h j is the bandwidth of a block j.
16. The system as claimed in claim 15 , wherein a mean shift vector is expressed as:
y
(
t
+
1
)
=
∑
i
=
1
N
w
i
(
α
i
(
u
i
,
t
)
x
i
-
v
i
(
u
i
,
t
)
)
∑
i
=
1
N
w
i
α
i
(
u
i
,
t
)
;
and wherein
α
i
(
u
,
t
)
=
∑
j
=
1
M
1
h
j
2
S
j
(
u
)
C
ij
(
t
)
and
v
i
(
u
,
t
)
=
∑
j
=
1
M
1
h
j
2
S
j
(
u
)
C
ij
(
t
)
z
_
j
.
17. The system as claimed in claim 12 , wherein the optimized bandwidth of a kernel is estimated by using motion capture.
18. The system as claimed in claim 12 , wherein the object is articulated.
19. The system as claimed in claim 12 , wherein the bandwidth of the plurality of kernels is determined by maximizing a functional of a class-separation metric.
20. The system as claimed in claim 12 , wherein the bandwidth of the plurality of kernels is determined by maximizing a functional of a class-discrimination metric.