Salient motion detection system, method and program product therefor
View Patent ↗An apparatus and method of identifying objects with salient motion in a scene and program product therefor. A region of change is identified within a scene. Optical flow is determined for the region of change. Pixels moving in a common direction are identified as seed pixels. Seed pixels are expanded to form pixel clusters. Temporal imaging results are fused to identify those objects with salient motion.
1. A method of identifying objects with salient motion in a scene, said method comprising of the steps of:
a) identifying a region of change within a scene, wherein identifying said region of change develops a temporal filter;
b) determining optical flow for said region;
c) identifying seed pixels moving in a common direction in said region;
d) expanding identified said seed pixels, pixel clusters being formed from expanded said seed pixels; and
e) fusing results of steps (a), (b) and (c), fused said results identifying objects having salient motion.
2. A method as in claim 1 , wherein said scene is a video stream including a plurality of frames, I(x, y, t).
3. A method as in claim 2 , wherein the step (a) of identifying said region of change comprises subtracting a subsequent image.
4. A method as in claim 2 , wherein the step (a) of identifying said region of change for each image comprises finding a difference image I difference (x, y, t) using a weighted accumulation, I accum (x, y, t), with a fixed weight for each new observation.
5. A method as in claim 2 , wherein said temporal filter is determined by calculating optical flow for said plurality of frames over a given period of time, [t, t+n].
6. A method as in claim 5 , wherein calculating said optical flow comprises identifying each of a plurality of pixels having a monotonously changing position.
7. A method as in claim 6 , wherein said plurality of said identified pixels in [t, t+n] comprise positive X-component pixels with positive X-component optical flow (P iX ), negative X-component with negative optical flow (N iX ), positive Y-component pixels with positive Y-component optical flow (P iY ) and negative Y-component with negative optical flow (N iY ).
8. A method as in claim 2 , wherein the step (d) of fusing results comprises combining an image of temporal difference (I differnce (x, y, t)) from step (a), temporal filtered images (I X-temporal (x, y, t) and I Y-temporal (x, y, t)) from step (b) and a region motion information (I RegionMotion (x, y, t)) from step (c), a salient motion image (I salient (x, y, t)) resulting from said combination having the form:
I salient ( x, y, t )= I difference ( x, y, t ) I [I X-temporal ( x, y, t ) Y I Y-temporal ( x, y, t ) Y I RegionMotion ( x, y, t )].
9. A computer program product for identifying objects with salient motion in a scene, said computer program product comprising a computer usable medium having computer readable program code thereon, said computer readable program code comprising:
computer readable program code means for identifying a region of change within a plurality of sequential frames, I(x, y, t), wherein identifying said region of change develops a temporal filter;
computer readable program code means for determining optical flow for said region of change;
computer readable program code means for identifying seed pixels in said region moving in a common direction;
computer readable program code means for expanding identified said seed pixels, pixel clusters being formed from expanded said seed pixels; and
computer readable program code means for fusing temporal imaging results.
10. A computer program product as in claim 9 , wherein the computer readable program code means for identifying said region of change for each image comprises computer readable program code means for subtracting a subsequent image.
11. A computer program product as in claim 9 , wherein the computer readable program code means for identifying said region of change for each image comprises computer readable program code means for finding a difference image I difference (x, y, t) using a weighted accumulation, I accum (x, y, t), with a fixed weight for each new observation.
12. A computer program product as in claim 10 , wherein computer readable program code means for developing said temporal filter comprises computer readable program code means for calculating optical flow for said plurality of frames over a given period of time, [t, t+n].
13. A computer program product as in claim 12 , wherein computer readable program code means for calculating said optical flow comprises computer readable program code means for identifying each of a plurality of pixels having a monotonously changing position.
14. A computer program product as in claim 13 , wherein said plurality of said identified pixels in [t, t+n] comprise positive X-component pixels with positive X-component optical flow (P iX ), negative X-component with negative optical flow (N iX ), positive Y-component pixels with positive Y-component optical flow (P iY ) and negative Y-component with negative optical flow (N iY ).
15. A computer program product as in claim 14 , wherein said computer readable program code means for identifying each of a plurality of pixels identifies pixels as belonging to an object with salient motion for a selected threshold T 1 and comprises computer readable program code means for:
representing optical flow of frames I t , I t+1 , . . . I t+n in [t, t+n] as F 1 , F 2 , . . . F n having an X-component, F 1,x , F 2,x , . . . F n,x and a Y-component is F 1,y , F 2,y , . . . F n,y , respectively, for i ⊂[1,n]; and, initially, P iX =0, and N iX =0; and otherwise
P
iX
=
{
P
(
i
-
1
)
X
+
1
,
if
(
F
i
,
X
>
0
)
P
(
i
-
1
)
X
,
otherwise
;
and
N
iX
=
{
N
(
i
-
1
)
X
+
1
,
if
(
F
i
,
X
<
0
)
N
(
i
-
1
)
X
,
otherwise
;
said identified pixels being identified by P nX ≧T 1 , N nX ≧T 1 , P nY ≧T 1 and N nY ≧T 1 .
16. An apparatus for identifying objects with salient motion in a scene, said apparatus comprising:
means for identifying a region of change within a plurality of sequential frames, I(x, y, t), wherein identifying said region of change develops a temporal filter;
means for determining optical flow for said region of change over a given period of time, [t, t+n];
means for identifying seed pixels in said region moving in a common direction;
means for expanding identified said seed pixels, pixel clusters being formed from expanded said seed pixels; and
means for fusing temporal imaging results.
17. A apparatus as in claim 16 , wherein the means for identifying said region of change for each image comprises:
means for finding a difference image I difference (x, y, t) using a weighted accumulation, I accum (x, y, t), with a fixed weight for each new observation.