Object activity modeling method
View Patent ↗An object activity modeling method which can efficiently model complex objects such as a human body is provided. The object activity modeling method includes the steps of (a) obtaining an optical flow vector from a video sequence; (b) obtaining the probability distribution of the feature vector for a plurality of video frames, using the optical flow vector; (c) modeling states, using the probability distribution of the feature vector; and (d) expressing the activity of the object in the video sequence based on state transition. According to the modeling method, in video indexing and recognition field, complex activities such as human activities can be efficiently modeled and recognized without segmenting objects.
1. An object activity modeling method comprising the steps of:
(a) obtaining an optical flow vector from a video sequence;
(b) obtaining a probability distribution of a feature vector for a plurality of video frames, using the optical flow vector, wherein the feature vector is an d×L dimensional vector, d being a number of dimensions and L being a number of pixels in a video frame or in a region of interest;
(c) modeling states, using the probability distribution of the feature vector; and
(d) expressing the activity of the object in the video sequence based on state transition.
2. The object activity modeling method of claim 1 , wherein the step (a) is based on affine motion estimation.
3. The object activity modeling method of claim 2 , wherein the step (a) further comprises the sub-steps of:
(a-1) grouping input video frames into a plurality of video frame groups and dividing each video frame group as an individual state;
(a-2) obtaining an affine motion parameter for each video in the video frame group of each individual state; and
(a-3) obtaining an optical flow vector from the affine motion parameters.
4. The object activity modeling method of claim 3 , wherein the step (a-2) comprises a step for determining parameters, which minimizes summed square difference Σ(I t (x)−I t−1 (x−V(x))) 2 over a given video based on the intensity of the pixel on the object, which is expressed as I t (x)=I t−1 (x−V(x)) when I denotes intensity, t denotes time, x denotes a pixel location (x, y), and v denotes the motion vector, as motion parameters.
5. The object activity modeling method of claim 1 , wherein the step (b) comprises a step for calculating probability distribution P(Z|Ω) by the following equation:
P
(
Z
❘
Ω
)
=
exp
(
-
1
2
(
z
-
m
)
T
)
Q
-
1
(
Z
-
m
)
)
(
2
π
)
N
Q
1
/
2
wherein P=(p 1 ,p 2 , . . . p d ) denotes a motion vector calculated at each pixel location (x, y), L denotes the number of pixels in a video frame or a region of interest, d denotes the number of dimensions, feature vector Z, which is a d×L dimension vector, is Z=(p 1 1 , p 1 2 , . . . , p 1 L , p 2 1 , p 2 2 , . . . , p 2 L , p d 1 , p d 2 , . . . p d L ) T , m is the mean vector of feature vector Z, and Q is the covariance matrix of feature vector Z, and it is assumed that feature vector Z is provided from observation class Ω.
6. The object activity modeling method of claim 1 , wherein the step (b) further comprises the steps of:
decomposing covariance matrix Q as the following equation:
Q=ΦΛΦ T
Wherein {circumflex over (Z)} is equal to Z−m, the columns of Φ are orthonormal eigenvectors of covariance matrix Q, and A corresponds to the diagonal eigenvalue; and
calculating probability distribution P(Z|Ω) by the following equation:
P
(
Z
❘
Ω
)
=
[
exp
(
-
1
2
∑
i
M
y
i
2
/
α
i
(
2
π
)
M
Λ
1
/
2
]
[
exp
(
-
1
2
∑
M
+
1
N
y
i
2
/
2
ρ
)
i
(
2
πρ
)
(
N
-
M
)
/
2
]
wherein M is the number of principal components, y i is the i-th component of Y, α i is the i-th eigenvalue of Q, and ρ is the optimal value, which is obtained by
ρ
=
1
N
-
M
∑
M
+
1
N
α
i
,
and it is assumed that feature vector Z is provided from observation class Ω.
7. The object activity modeling method of claim 1 , wherein in the step (c), the object activity in the video sequence is expressed using a Hidden Markov Model (HMM), based on state transition.
8. The object activity modeling method of claim 7 , wherein the Hidden Markov Model (HMM) is expressed as λ={Ξ, A, B, Π} when N is the number of possible states, Ξ satisfies Ξ={q 1 , q 2 , . . . , q N }, A is {a ij }, the transition between hidden states i and j, B is {b j (.)}, the observation symbol probability corresponding to state j, and Π is the initial state distribution, and the state Ξ={q 1 , q 2 , . . . , q N } and the initial state distribution Π are determined in advance based on video data.