System and method for moving object detection and processing

1. A method for an intelligent video processing system based on object detection, comprising:

receiving an input video sequence corresponding to a video program;

obtaining a plurality of frames of the input video sequence;

obtaining a computational constraint and a temporal rate constraint;

determining one or more regions of interest (ROIs) of the plurality of frames of the input video sequence based on the computational constraint and temporal rate constraint;

selecting a desired set of frames from the plurality of frames based on the ROIs such that the desired set of frames substantially represent a view path of the plurality of frames; and

detecting object occurrences from the desired set of frames based on the selected desired set of frames such that a computational cost and a number of frames for detecting the object occurrences are under the computational constraint and temporal rate constraint;

wherein:

the computational constraint specifies an upper bound on a total processing time consumed by determining the one or more ROIs, selecting the desired set of frames and detecting the object occurrences; and

the computational constraint and the temporal rate constraint are automatically determined according to operational parameters and computational capacity of the intelligent video processing system.

2. The method according to claim 1 , wherein determining the ROIs of the plurality of frames further includes:

computing the motion and texture information of the plurality of frames.

3. The method according to claim 2 , wherein determining the ROIs of the plurality of frames further includes:

decomposing the motion into motion angles and magnitude values;

based on the motion angles and magnitude values, using a block labeling algorithm to select ROIs such that the selected ROIs coarsely enclose zones of salient motion and relevant texture of the plurality of frames.

4. The method according to claim 3 , wherein using the block labeling algorithm further includes:

when a block is with zero motion, not labelling the block as having local motion;

when a block is with no texture or no salient motion in its neighborhoods, not labelling the block as having local motion; and

when a roughness of a block is lower than a threshold, not labelling the block as having foreground motion.

5. The method according to claim 1 , wherein,

provided that S represents the plurality of frames, N represents a total number of frames of the plurality of frames, Q(S) represents quality of S in representing the view path, D o (S) is an object detector of S, R C D (S) represents a computational allowance to be consumed by executing D o (S), R T (S) represents a temporal cost of S, R C R (V) represents a computational allowance already consumed in determining the ROIs for the plurality of frames, γ represents the computational constraint, and κ represents the temporal rate constraint, selecting the desired set of frames further includes:

selecting the desired set of frames by solving an optimization problem as:

maximize Q ( S ) subject to R C D ( S )≦γ N−R C R ( V ) and R T ( S )≦κ N.

6. The method according to claim 5 , further including:

using a dynamic programming algorithm to solve the optimization problem.

7. The method according to claim 6 , further including:

utilizing local representation and content coverage information to represent quality of potential frame sets;

creating links between possible adjacent frames in each frame set based on the quality of each frame set;

creating a directed acyclic graph (DAG); and

solving a longest path problem in the DAG using the dynamic programming algorithm.

8. The method according to claim 1 , wherein detecting the object occurrences further includes:

executing an object detector over the selected desired set of frames to detect the object occurrences.

9. An intelligent video processing system, comprising:

a video decoder configured to decode an incoming video program; and

a processor configured to:

obtain a plurality of frames of the input video sequence;

obtain a computational constraint and a temporal rate constraint;

determine one or more regions of interest (ROIs) of the plurality of frames of the input video sequence based on the computational constraint and temporal rate constraint;

select a desired set of frames from the plurality of frames based on the ROIs such that the desired set of frames substantially represent a view path of the plurality of frames; and

detect object occurrences from the desired set of frames based on the selected desired set of frames such that a computational cost and a number of frames for detecting the object occurrences are under the computational constraint and temporal rate constraint;

wherein:

the computational constraint specifies an upper bound on a total processing time consumed by determining the one or more ROIs, selecting the desired set of frames and detecting the object occurrences; and

the computational constraint and the temporal rate constraint are automatically determined according to operational parameters and computational capacity of the intelligent video processing system.

10. The intelligent video processing system according to claim 9 , wherein, to determine the ROIs of the plurality of frames, the processor is further configured to:

compute the motion and texture information of the plurality of frames.

11. The intelligent video processing system according to claim 10 , wherein, to determine the ROIs of the plurality of frames, the processor is further configured to:

decompose the motion into motion angles and magnitude values;

based on the motion angles and magnitude values, use a block labeling algorithm to select ROIs such that the selected ROIs coarsely enclose zones of salient motion and relevant texture of the plurality of frames.

12. The intelligent video processing system according to claim 11 , wherein, to use the block labeling algorithm, the processor is further configured:

when a block is with zero motion, not to label the block as having local motion;

when a block is with no texture or no salient motion in its neighborhoods, not to label the block as having local motion; and

when a roughness of a block is lower than a threshold, not to label the block as having foreground motion.

13. The intelligent video processing system according to claim 9 , wherein, to select the desired set of frames, the processor is further configured to:

provided that S represents the plurality of frames, N represents a total number of frames of the plurality of frames, Q(S) represents quality of S in representing the view path, D o (S) is an object detector of S, R C D (S) represents a computational allowance to be consumed by executing D o (S), R T (S) represents a temporal cost of S, R C R (V) represents a computational allowance already consumed in determining the ROIs for the plurality of frames, γ represents the computational constraint, and κ represents the temporal rate constraint,

select the desired set of frames by solving an optimization problem as:

maximize Q ( S ) subject to R C D ( S )≦γ N−R C R ( V ) and R T ( S )≦κ N.

14. The intelligent video processing system according to claim 13 , the processor is further configured to:

use a dynamic programming algorithm to solve the optimization problem.

15. The intelligent video processing system according to claim 14 , the processor is further configured to:

utilize local representation and content coverage information to represent quality of potential frame sets;

create links between possible adjacent frames in each frame set based on the quality of each frame set;

create a directed acyclic graph (DAG); and

solve a longest path problem in the DAG using the dynamic programming algorithm.

16. The intelligent video processing system according to claim 9 , wherein, to detect the object occurrences, the processor is further configured to:

execute an object detector over the selected desired set of frames to detect the object occurrences.