Detection of abnormal crowd behavior
View Patent ↗A system and method detects the intent and/or motivation of two or more persons or other animate objects in a video scene. In one embodiment, the system forms a blob of the two or more persons, draws a bounding box around said blob, calculates an entropy value for said blob, and compares that entropy value to a threshold to determine if the two or more persons are involved in a fight or other altercation.
1. A method comprising a computer to perform:
tracking two or more animate objects in a video scene;
forming a single blob by combining into the single blob image data corresponding to said two or more animate objects;
forming a bounding box around said blob; and
calculating a spatial and temporal entropy value for said bounding box.
2. The method of claim 1 , further comprising determining whether said entropy value is greater than a threshold.
3. The method of claim 2 , wherein said determining whether said entropy value is greater than said threshold indicates that said two or more animate objects are involved in a fight.
4. The method of claim 1 , further comprising determining whether said entropy value is within a certain percentage of a threshold.
5. The method of claim 1 , further comprising setting a region of interest around said bounding box.
6. The method of claim 5 , wherein said region of interest is approximately 25% greater than said bounding region.
7. The method of claim 6 , wherein said entropy value is calculated for said region of interest.
8. The method of claim 1 , further comprising normalizing said entropy value.
9. The method of claim 8 , wherein said normalized entropy is calculated on a per pixel basis as follows:
D
(
x
,
y
,
t
)
=
{
1
if
I
(
x
,
y
,
t
)
-
I
(
x
,
y
,
t
-
τ
)
>
ɛ
0
otherwise
wherein
D(x, y, t) represents a change in intensity of a pixel;
ε is a statistical variance; and
τ is a time interval; and further wherein the normalized entropy comprises:
entropy
=
1
T
1
S
B
∑
t
∑
(
x
,
y
)
∈
B
D
(
x
,
y
,
t
)
wherein
S B denotes a size of a region of interest; and
T denotes a temporal duration of the entropy that is calculated after normalization.
10. The method of claim 8 , wherein said normalized entropy is calculated on a per sub-window basis as follows:
m
i
(
t
)
=
1
n
pixel
∑
(
x
,
y
)
∈
Window_i
I
(
x
,
y
,
t
)
wherein
m i (t) is the mean intensity value of a sub-window i at time t; and further wherein
D
*
(
i
,
t
)
=
{
1
if
m
i
(
t
)
-
m
i
(
t
-
τ
)
>
ɛ
*
0
otherwise
wherein
ε is a statistical variance; and
τ is a time interval; and further wherein
entropy
=
1
T
1
n
sub
-
window
∑
i
D
*
(
i
,
t
)
wherein
T denotes a temporal duration of the entropy that is calculated after normalization; and
n sub-window denotes the number of sub-windows.
11. The method of claim 1 , further comprising determining the real world location of said two or more animate objects in said video scene.
12. The method of claim 1 , wherein said entropy value comprises a mean intensity.
13. A system comprising a computer to perform:
a tracking module to track two or more animate objects in a video scene;
a module to form a single blob by combining into the single blob image data corresponding to said two or more animate objects;
a module to form a bounding box around said blob; and
a calculation module to calculate a spatial and temporal entropy value for said bounding box.
14. The system of claim 13 , further comprising a module to:
set a region of interest around said blob;
calculate said entropy value for said region of interest; and
normalize said entropy value.
15. The system of claim 13 , wherein said module to normalize said entropy value is on a pixel basis and comprises:
D
(
x
,
y
,
t
)
=
{
1
if
I
(
x
,
y
,
t
)
-
I
(
x
,
y
,
t
-
τ
)
>
ɛ
0
otherwise
wherein
D(x, y, t) represents a change in intensity of a pixel;
ε is a statistical variance; and
τ is a time interval; and further wherein the normalized entropy comprises:
entropy
=
1
T
1
S
B
∑
t
∑
(
x
,
y
)
∈
B
D
(
x
,
y
,
t
)
wherein
S B denotes a size of a region of interest; and
T denotes a temporal duration of the entropy that is calculated after normalization.
16. The system of claim 13 , wherein said module to normalize said entropy value is on a sub-window basis and comprises:
m
i
(
t
)
=
1
n
pixel
∑
(
x
,
y
)
∈
Window_i
I
(
x
,
y
,
t
)
wherein
m i (t) is the mean intensity value of a sub-window i at time t; and further wherein
D
*
(
i
,
t
)
=
{
1
if
m
i
(
t
)
-
m
i
(
t
-
τ
)
>
ɛ
*
0
otherwise
wherein
ε is a statistical variance; and
τ is a time interval; and further wherein
entropy
=
1
T
1
n
sub
-
window
∑
i
D
*
(
i
,
t
)
wherein
T denotes a temporal duration of the entropy that is calculated after normalization; and
n sub-window denotes the number of sub-windows.
17. A machine readable medium having stored instructions thereon for executing a process comprising:
tracking two or more animate objects in a video scene;
forming a single blob by combining into the single blob image data corresponding to said two or more animate objects;
forming a bounding box around said blob; and
calculating a spatial and temporal entropy value for said bounding box.
18. The machine readable medium of claim 17 , further comprising comparing said entropy value to a threshold to determine whether said entropy value is greater than said threshold or said entropy value is within a certain percentage of said threshold.
19. The machine readable medium of claim 17 , wherein said entropy value is normalized on a per pixel basis and comprises:
D
(
x
,
y
,
t
)
=
{
1
if
I
(
x
,
y
,
t
)
-
I
(
x
,
y
,
t
-
τ
)
>
ɛ
0
otherwise
wherein
D(x, y, t) represents a change in intensity of a pixel;
ε is a statistical variance; and
τ is a time interval; and further wherein the normalized entropy comprises:
entropy
=
1
T
1
S
B
∑
t
∑
(
x
,
y
)
∈
B
D
(
x
,
y
,
t
)
wherein
S B denotes a size of a region of interest; and
T denotes a temporal duration of the entropy that is calculated after normalization.
20. The machine readable medium of claim 17 , wherein said entropy value is normalized on a per sub-window basis and comprises:
m
i
(
t
)
=
1
n
pixel
∑
(
x
,
y
)
∈
Window_i
I
(
x
,
y
,
t
)
wherein
m i (t) is the mean intensity value of a sub-window i at time t; and further wherein
D
*
(
i
,
t
)
=
{
1
if
m
i
(
t
)
-
m
i
(
t
-
τ
)
>
ɛ
*
0
otherwise
wherein
ε is a statistical variance; and
τ is a time interval; and further wherein
entropy
=
1
T
1
n
sub
-
window
∑
i
D
*
(
i
,
t
)
wherein
T denotes a temporal duration of the entropy that is calculated after normalization; and
n sub-window denotes the number of sub-windows.