Object tracker, object tracking method, and computer program
There is provided an object tracking apparatus that realizes robust object detection and tracking even for movement fluctuation and observation noise, an object tracking method and a computer program. An object cracking apparatus 1 is an apparatus tracking an object in video, the object tracking apparatus 1 including: a deep learning discriminator 2 which is a discriminator by deep learning; and a particle filter function unit 3 tracking an object by applying a multi-channel feature value of video including feature values by the deep learning discriminator 2 to likelihood evaluation by a particle filter, according to a distance between position information about the multi-channel feature value and position information about each particle.
1. An object tracking apparatus tracking an object in video, comprising:
a deep learning discriminator that is implemented in one or more coputers and that is a discriminator by deep learning; and
a particle filter function unit that is implemented in one or more computers and that is configured to track the object by applying a multi-channel feature value of video including feature values to the deep learning discriminator for likelihood evaluation using a particle filter, according to a distance between position information about the multi-channel feature value and position information about each particle.
2. The object tracking apparatus according to claim 1 , wherein the particle filter function unit is configured to dynamically change weighting of likelihood of each particle according to a state of a scene, based on the multi-channel feature value.
3. The object tracking apparatus according to claim 1 , wherein the particle filter function unit is configured to create, for each object, a feature value histogram for the tracked object at time t by (i) extracting only particles having a label corresponding to a label of a rectangle of the object detected by the deep learning discriminator and existing in the rectangle, (ii) extracting pixel areas around the particles, and (iii) calculating feature values for the pixel areas, and obtain a likelihood function for the tracked object at the time t by determining a weighted sum of a histogram at time t−1 and the histogram at the time t.
4. The object tracking apparatus according to claim 3 , wherein, based on a certain particle existing in the rectangle detected by the deep learning discriminator, the particle filter function unit is configured to update likelihood of the particle by the following formula, based on reliability of the rectangle detected by the deep learning discriminator being indicated by p, a distance from a rectangle center to four corners being indicated by R, a distance to the particle being indicated by r, the likelihood of the particle at the time t being indicated by π t , and a parameter to determine a contribution rate of the deep learning discriminator being indicated by β,
π
t
←
π
t
(
1
+
p
λ
t
)
β
[
Formula
1
]
λ
t
=
max
{
0
,
1
-
r
2
R
2
}
.
[
Formula
2
]
5. The object tracking apparatus according to claim 4 , wherein the particle filter function unit is configured to evaluate the particle as “having more objectness” based on the reliability p of a detection result of the deep learning discriminator being higher and a position of the particle being closer to the rectangle center.
6. The object tracking apparatus according to claim 1 , further comprising a sensor unit, wherein the particle filter function unit is configured to combine sensor data from the sensor unit with the likelihood evaluation by the particle filter, in addition to the object detection result by the deep learning discriminator.
7. An object tracking method for tracking an object in video, comprising:
detecting the object; and
tracking the object by applying a multi-channel feature value of video including feature values to a deep learning discriminator for likelihood evaluation using a particle filter, according to a distance between position information about the multi-channel feature value and position information about each particle.
8. A non-transitory computer readable medium storing a computer program, wherein execution of the computer program causes one or more computers perform operations comprising:
detecting an object in video; and
tracking the object by applying a multi-channel feature value of video including feature values to a deep learning discriminator for likelihood evaluation using a particle filter, according to a distance between position information about the multi-channel feature value and position information about each particle.
9. The non-transitory computer readable medium according to claim 8 , wherein the operations further comprise dynamically changing weighting of likelihood of each particle according to a state of a scene, based on the multi-channel feature value.
10. The non-transitory computer readable medium according to claim 8 , wherein the operations further comprise:
creating, for each object, a feature value histogram for the tracked object at time t by (i) extracting only particles having a label corresponding to a label of a rectangle of the object detected by the deep learning discriminator and existing in the rectangle, (ii) extracting pixel areas around the particles, and (iii) calculating feature values for the pixel areas; and
obtaining a likelihood function for the tracked object at the time t by determining a weighted sum of a histogram at time t−1 and the histogram at the time t.
11. The non-transitory computer readable medium according to claim 10 , wherein the operations further comprise, based on a certain particle existing in the rectangle detected by the deep learning discriminator, updating likelihood of the particle by the following formula, based on reliability of the rectangle detected by the deep learning discriminator being indicated by p, a distance from a rectangle center to four corners being indicated by R, a distance to the particle being indicated by r, the likelihood of the particle at the time t being indicated by π t , and a parameter to determine a contribution rate of the deep learning discriminator being indicated by β,
π
t
←
π
t
(
1
+
p
λ
t
)
β
[
Formula
1
]
λ
t
=
max
{
0
,
1
-
r
2
R
2
}
.
[
Formula
2
]
12. The non-transitory computer readable medium according to claim 11 , wherein the operations further comprise evaluating the particle as “having more objectness” based on the reliability p of a detection result of the deep learning discriminator being higher and a position of the particle being closer to the rectangle center.
13. The non-transitory computer readable medium according to claim 8 , wherein the operations further comprise combining sensor data from a sensor unit with the likelihood evaluation by the particle filter, in addition to the object detection result by the deep learning discriminator.