Methods for recognizing pose and action of articulated objects with collection of planes in motion
View Patent ↗The invention comprises an improved system, method, and computer-readable instructions for recognizing pose and action of articulated objects with collection of planes in motion. The method starts with a video sequence and a database of reference sequences corresponding to different known actions. The method identifies the sequence from the reference sequences such that the subject in performs the closest action to that observed. The method compares actions by comparing pose transitions. The cross-homography invariant may be used for view-invariant recognition of human body pose transition and actions.
1. A computer-implemented method for recognizing pose and action of articulated objects, comprising:
(a) receiving by a computing device action sequence information as a set of points for each of a plurality of images obtained from a single camera view, wherein the set of points represents a pose P of an articulated object defined by M body points obtained by using an articulated object tracking technique;
(b) processing the action sequence information by dividing the set of points into triplets of points, wherein each triplet of non-collinear points defines a plane in 3D space, and determining a pose transition from a rigid motion of the plane in 3D space;
(c) finding one or more probable matches by comparing the pose transition of the action sequence information to a pose transition of a reference sequence; and
(d) outputting at least one of the one or more probable matches;
wherein invariants of the planes are found for the rigid motion of the planes across two frames; and
wherein the invariants of the planes are found by one or both of a vector of fundamental ratios and an equality of eigenvalues of cross-homography, thereby providing a view-invariant recognition of actions.
2. The computer-implemented method of claim 1 wherein the action sequence information is received in real time or from previously recorded action sequence information.
3. The computer-implemented method of claim 1 wherein the reference sequence comprises a sequence of two-dimensional (2D) poses for each of a known action maintained in an action database of known actions.
4. The computer-implemented method of claim 1 wherein the pose P is represented by M body points such that P={mi=1 . . . M}.
5. The computer-implemented method of claim 1 further comprising calculating and outputting by the processing device confidence level information for the one or more probable matches.
6. The computer-implemented method of claim 1 wherein the vector of fundamental ratios defines three projective invariants for a configuration involving a moving plane as a moving triplet of points as observed in two frames by a stationary camera.
7. The computer-implemented method of claim 1 wherein the output of at least one of the one or more probable matches is utilized in one or more of a video surveillance system or a gesture-based computer interface.
8. A non-transitory computer readable medium containing program instructions for execution on a computing device, which when executed by the computing device, causes the computing device to:
(a) receive action sequence information as a set of points for each of a plurality of images obtained from a single camera view, wherein the set of points represents a pose P of an articulated object defined by M body points obtained by using an articulated object tracking technique;
(b) process the action sequence information by dividing the set of points into triplets of points, wherein each triplet of non-collinear points defines a plane in 3D space, and determining a pose transition from a rigid motion of the plane in 3D space;
(c) finding one or more probable matches by comparing the pose transition of the action sequence information to a pose transition of a reference sequence; and
(d) output at least one of the one or more probable matches;
wherein invariants of the planes are found for the rigid motion of the planes across two frames; and
wherein the invariants of the planes are found by one or both of a vector of fundamental ratios and an equality of eigenvalues of cross-homography, thereby providing a view-invariant recognition of actions.
9. The non-transitory computer readable medium of claim 8 further comprising calculating and outputting by the processing device confidence level information for the one or more probable matches.
10. The non-transitory computer readable medium of claim 8 wherein the vector of fundamental ratios defines three projective invariants for a configuration involving a moving plane as a moving triplet of points as observed in two frames by a stationary camera.
11. A system for recognizing pose and action of articulated objects, comprising:
a computing device having at least one processing device, a memory coupled to the processing device, the computing device in communication with a data source comprising action sequence information of an articulated object and a database of known actions; wherein the processing device executes a program residing in the memory to:
(a) receive the action sequence information from the data source as a set of points for each of a plurality of images obtained from a single camera view, wherein the set of points represents a pose P of an articulated object defined by M body points obtained by using an articulated object tracking technique;
(b) process the action sequence information by dividing the set of points into triplets of points, wherein each triplet of non-collinear points defines a plane in 3D space, and determining a pose transition from a rigid motion of the plane in 3D space;
(c) find one or more probable matches by comparing the pose transition of the action sequence information to a pose transition of a reference sequence; and
(d) output at least one of the one or more probable matches;
wherein invariants of the planes are found for the rigid motion of the planes across two frames; and
wherein the invariants of the planes are found by one or both of a vector of fundamental ratios and an equality of eigenvalues of cross-homography, thereby providing a view-invariant recognition of actions.
12. The system of claim 11 further comprising calculating and outputting by the processing device confidence level information for the one or more probable matches.
13. The system of claim 11 wherein the vector of fundamental ratios defines three projective invariants for a configuration involving a moving plane as a moving triplet of points as observed in two frames by a stationary camera.