Method and apparatus to segment motion area in real-time to detect motion in surveillance camera system
View Patent ↗A method and apparatus to segment a motion area in real-time to detect motion in a surveillance camera system are provided. The method includes updating a background image by using a previous input image from among an input image sequence, generating a difference image between a current image of the image sequence and the background image, generating a second function to minimize a first function including regularized energy in the motion area of the difference image and regularized energy in an area without motion of the difference image and segmenting the motion area based on the second function. Therefore, while noise is removed, the motion area is segmented accurately and rapidly even in a low illumination environment so as to detect a moving object.
1. A method of segmenting a motion area from a current image, the method comprising:
obtaining a background image of the current image by using a previous image of the current image;
obtaining a difference image between the current image and the background image;
obtaining a level set function of the current image to minimize an energy function of the current image comprising a regularized energy of the motion area of the difference image and a regularized energy of a non-motion area of the difference image; and
segmenting the motion area from the current image based on the level set function,
wherein the regularized energy of the motion area is obtained by using a difference between a first standard value generated by using the difference image, corresponding to an average pixel value of the motion area, and the difference image, and the regularized energy of the non-motion area is obtained by using a difference between a second standard value generated by using the difference image, corresponding to an average pixel value of the non-motion area, and the difference image, and
wherein the obtaining the level set function comprises iteratively calculating the level set function n+1 times, where n is an integer equal to or greater than 1, and the level set function obtained at a k-th calculation is calculated based on a level set function obtained at a (k−1)-th calculation, the difference image, the first and second standard values used to obtain the level set function obtained at the (k−1)-th calculation, where k is an integer equal to or greater than 2 and equal to or less than n+1, and
wherein, by the iteratively calculating, the level set function of the current image and the first and second standard values of the current image change at each time of the iterative calculating.
2. The method of claim 1 , wherein the obtaining a background image comprises adding a result obtained by multiplying the previous image by a first weight to a result obtained by multiplying a background image of the previous image by a second weight, wherein a sum of the first weight and the second weight is 1.
3. The method of claim 1 , wherein the segmenting the motion area from the current image comprises:
determining a pixel of the difference image in which the level set function is greater than 0 as a pixel of the motion area;
determining a pixel of the difference image in which the level set function is less than 0 as a pixel of the non-motion area;
determining a pixel of the difference image in which the level set function is 0 as a pixel of a boundary of the motion area; and
when a number of pixels in which the level set function is greater than 0 is above a predetermined threshold, determining that motion is detected in the current image.
4. The method of claim 1 , wherein the energy function comprises noise energy using a function related to a partial differential value of the level set function.
5. The method of claim 1 , wherein
the integer n is determined based on a set value to which the level set function converges.
6. The method of claim 1 , wherein the level set function obtained by the iterative calculating is a real time value with respect to the current image.
7. The method of claim 1 , wherein each of the regularized energies of the motion area and the non-motion area is regularized by being divided by a sum of the difference between the difference image and the first standard value, and the difference between the difference image and the second standard value, respectively.
8. The method of claim 1 , further comprising:
determining a noise level in the current image, and
adjusting the first standard value by a weight according to the determining.
9. The method of claim 1 , wherein an initial value of the level set function of the current image for the iterative calculating is set to the second standard value of the previous image.
10. The method of claim 1 , wherein the segmenting the motion area from the current image comprises determining that a sudden light change exists if a difference between the second standard value and a second standard value corresponding to an average pixel value of a non-motion area of the previous image is above a predetermined threshold.
11. The method of claim 1 , wherein the energy function is represented by an equation of:
E
(
ϕ
)
=
∫
Ω
Φ
(
∇
ϕ
)
ⅆ
x
+
λ
1
∫
Ω
H
(
α
+
ϕ
)
(
u
(
x
)
-
P
+
(
ϕ
,
u
)
)
2
ϕ
(
x
)
λ
1
(
u
(
x
)
-
P
+
(
ϕ
,
u
)
)
2
+
λ
2
(
u
(
x
)
-
P
-
(
ϕ
,
u
)
)
2
ⅆ
x
-
λ
2
∫
Ω
H
(
α
-
ϕ
)
(
u
(
x
)
-
P
-
(
ϕ
,
u
)
)
2
ϕ
(
x
)
λ
1
(
u
(
x
)
-
P
+
(
ϕ
,
u
)
)
2
+
λ
2
(
u
(
x
)
-
P
-
(
ϕ
,
u
)
)
2
ⅆ
x
,
where E(φ) is the energy function, φ is the level set function, u is the difference image, P + (φ, u) and P − (φ, u) are the first and second standard values respectively in areas of the current image where the level set function φ is greater than 0 and less than 0, respectively, depending on the level set function φ and the difference image u, H(s) a step function, wherein H(s)=1 when s≧0 and H(s)=0 when s<0, Φ is a predetermined function, and α, λ 1 , and λ 2 are variables in which the difference image is considered,
E
(
ϕ
)
=
∫
Ω
Φ
(
∇
ϕ
)
ⅆ
x
-
λ
2
∫
Ω
H
(
α
-
ϕ
)
(
u
(
x
)
-
P
-
(
ϕ
,
u
)
)
2
ϕ
(
x
)
λ
1
(
u
(
x
)
-
P
+
(
ϕ
,
u
)
)
2
+
λ
2
(
u
(
x
)
-
P
-
(
ϕ
,
u
)
)
2
ⅆ
x
.
12. The method of claim 11 , wherein the level set function φ is obtained by calculating equation of:
∂
∂
t
ϕ
(
x
,
t
)
=
∇
·
(
Φ
′
(
∇
ϕ
)
∇
ϕ
∇
ϕ
)
+
λ
2
(
u
(
x
)
-
P
-
(
ϕ
,
u
)
)
2
(
H
(
α
-
ϕ
)
-
ϕ
H
′
(
α
-
ϕ
)
)
(
u
(
x
)
-
P
+
(
ϕ
,
u
)
)
2
+
(
u
(
x
)
-
P
-
(
ϕ
,
u
)
)
2
ϕ
i
,
j
n
+
1
=
ϕ
i
,
j
n
+
1
4
{
ϕ
i
-
1
,
j
n
+
ϕ
i
+
1
,
j
n
+
ϕ
i
,
j
-
1
n
+
ϕ
i
,
j
+
1
n
-
4
ϕ
i
,
j
n
}
+
-
λ
1
u
-
P
+
(
ϕ
n
,
u
)
2
+
λ
2
u
-
P
-
(
ϕ
n
,
u
)
2
λ
1
u
-
P
+
(
ϕ
n
,
u
)
2
+
λ
2
u
-
P
-
(
ϕ
n
,
u
)
2
,
where φ i,j n+1 is a level set function obtained at an (n+1)th calculation based on a level set function φ i,j n obtained at an n-th calculation, and i and j indicate a coordinate of a pixel in the current image.
13. An apparatus to segment a motion area from a current image, the apparatus comprising:
an input device which receives the current image;
a background image generation device which obtains a background image of the current image by using a previous image of the current image;
a difference image generation device which obtains a difference image between the current image and the background image;
a function generation device which obtains a level set function of the current image to minimize an energy function of the current image, the energy function comprising a regularized energy of the motion area of the difference image and a regularized energy of a non-motion area of the difference image; and
a motion segmentation device which segments the motion area from the current image based on the level set function,
wherein the function generation device obtains the regularized energy of the motion area by using a difference between a first standard value generated by using the difference image, corresponding to an average pixel value of the motion area, and the difference image, and obtains the regularized energy of the non-motion area by using a difference between a second standard value generated by using the difference image, corresponding to an average pixel value of the non-motion area, and the difference image, and
wherein the function generation device obtains the level set function by iteratively calculating the level set function n+1 times, where n is an integer equal to or greater than 1, and the level set function obtained at a k-th calculation is calculated based on a level set function obtained at a (k−1)-th calculation, the difference image, the first and second standard values used to obtain the level set function obtained at the (k−1)-th calculation, where k is an integer equal to or greater than 2 and equal to or less than n+1, and
wherein, by the iteratively calculating, the level set function of the current image and the first and second standard values of the current image change at each time of the iterative calculating.
14. The apparatus of claim 13 , wherein
the integer n is determined based on a set value to which the level set function converges,
wherein the level set function obtained by the iterative calculating is a real time value with respect to the current image, and
wherein the function generation device determines a noise level in the current image, and adjusts the first standard value by a weight according to the noise level.
15. The apparatus of claim 13 , wherein, in the background image generation device, a result obtained by multiplying the previous image by a first weight is added to a result obtained by multiplying a background image of the previous image by a second weight, wherein a sum of the first weight and the second weight is 1.
16. The apparatus of claim 13 , wherein the motion segmentation device
determines a pixel of the difference image in which the level set function is greater than 0 as a pixel of the motion area, a pixel of the difference image in which the level set function is less than 0 as a pixel of the non-motion area, and a pixel of the difference image in which the level set function is 0 as a pixel of a boundary of the motion area,
wherein, the motion segmentation device further determines that motion is detected in the current image, when a number of pixels in which the level set function value is greater than 0 is above a predetermined threshold.
17. The apparatus of claim 13 , wherein
an initial value of the level set function of the current image for the iterative calculating is set to the second standard value of the previous image.
18. The apparatus of claim 13 , wherein the motion segmentation device determines that a sudden light change exists if a difference between the second standard value and a second standard value corresponding to an average pixel value of a non-motion area of the previous image is above a predetermined threshold.
19. The apparatus of claim 13 , wherein the energy function is represented by an equation of:
E
(
ϕ
)
=
∫
Ω
Φ
(
∇
ϕ
)
ⅆ
x
+
λ
1
∫
Ω
H
(
α
+
ϕ
)
(
u
(
x
)
-
P
+
(
ϕ
,
u
)
)
2
ϕ
(
x
)
λ
1
(
u
(
x
)
-
P
+
(
ϕ
,
u
)
)
2
+
λ
2
(
u
(
x
)
-
P
-
(
ϕ
,
u
)
)
2
ⅆ
x
-
λ
2
∫
Ω
H
(
α
-
ϕ
)
(
u
(
x
)
-
P
-
(
ϕ
,
u
)
)
2
ϕ
(
x
)
λ
1
(
u
(
x
)
-
P
+
(
ϕ
,
u
)
)
2
+
λ
2
(
u
(
x
)
-
P
-
(
ϕ
,
u
)
)
2
ⅆ
x
,
where E(φ) is the energy function, φ is the level set function, u is the difference image, P + (φ, u) and P − (φ, u) are the first and second standard values respectively in areas of the current image where the level set function φ is greater than 0 and less than 0, respectively, depending on the level set function φ and the difference image u, H(s) is a step function, wherein H(s)=1 when s≧0 and H(s)=0 when s<0, Φ is a predetermined function, and α, λ 1 , and λ 2 are variables in which the difference image is considered,
E
(
ϕ
)
=
∫
Ω
Φ
(
∇
ϕ
)
ⅆ
x
-
λ
2
∫
Ω
H
(
α
-
ϕ
)
(
u
(
x
)
-
P
-
(
ϕ
,
u
)
)
2
ϕ
(
x
)
λ
1
(
u
(
x
)
-
P
+
(
ϕ
,
u
)
)
2
+
λ
2
(
u
(
x
)
-
P
-
(
ϕ
,
u
)
)
2
ⅆ
x
.
20. The method of claim 13 , wherein each of the regularized energies of the motion area and the non-motion area is regularized by being divided by a sum of the difference between the difference image and the first standard value, and the difference between the difference image and the second standard value, respectively.
21. A computer readable recording medium having embodied thereon a computer program to execute a method, the method comprising:
obtaining a background image of the current image by using a previous image of the current image;
obtaining a difference image between the current image and the background image;
obtaining a level set function of the current image to minimize an energy function of the current image comprising a regularized energy of the motion area of the difference image and a regularized energy of a non-motion area of the difference image; and
segmenting the motion area from the current image based on the level set function,
wherein the regularized energy of the motion area is obtained by using a difference between a first standard value generated by using the difference image, corresponding to an average pixel value of the motion area, and the difference image, and the regularized energy of the non-motion area is obtained by using a difference between a second standard value generated by using the difference image, corresponding to an average pixel value of the non-motion area, and the difference image, and
wherein the obtaining the level set function comprises iteratively calculating the level set function n+1 times, where n is an integer equal to or greater than 1, and the level set function obtained at a k-th calculation is calculated based on a level set function obtained at a (k−1)-th calculation, the difference image, the first and second standard values used to obtain the level set function obtained at the (k−1)-th calculation, where k is an integer equal to or greater than 2 and equal to or less than n+1, and
wherein, by the iteratively calculating, the level set function of the current image and the first and second standard values of the current image change at each time of the iterative calculating.