Motion identification method and system
The present invention provides an action recognition method and system thereof. The action recognition method comprises: capturing a 2D image and a depth image at the same time, extracting an 2D information of the human skeleton points from the 2D image and correcting it, mapping the 2D information of the human skeleton points to the depth image to obtain the corresponding depth information with respect to the 2D information of the human skeleton points and combining the corrected 2D information of the human skeleton points and the depth information to obtain the 3D information of the human skeleton points, and finally recognizing an action from a set of 3D information of the human skeleton points during a period of time by a matching model.
1. A motion identification method, comprising
capturing a 2D color image or a 2D infrared image and a corresponding depth image at a time point;
extracting a 2D human skeleton point information from the 2D color image or the 2D infrared image;
mapping the 2D human skeleton point information to the depth image to obtain a depth information corresponding to the 2D human skeleton point information;
correcting the 2D human skeleton point information using a size-depth parameter and a distortion model;
combining the corrected 2D human skeleton point information and the depth information to obtain a 3D human skeleton point information; and
applying a match model to a series of the 3D human skeleton point information in a period of time to identify a motion.
2. The motion identification method of claim 1 , further comprising: transmitting an alert signal while the motion is identified.
3. The motion identification method of claim 1 , wherein the distortion model is used to correct the distance between the pixel coordinate position of the 2D human skeleton point and the image distortion center.
4. The motion identification method of claim 1 , wherein the match model is a classification model parameter established by a deep learning framework of neural network.
5. The motion identification method of claim 1 , wherein the depth image is corrected with a displacement parameter in advance.
6. A motion identification system, comprising
an image capturing device for capturing a 2D color image or a 2D infrared image at a time point;
a depth image capturing device for capturing a corresponding depth image at the time point;
a memory for storing a size-depth parameter, a distortion model, and a match model; and
a processor electrically connected to the image capturing device, the depth image capturing device and the memory, the processor comprising:
an input module for receiving the 2D color image or the 2D infrared image and the corresponding depth image;
a storage module for storing the 2D color image or the 2D infrared image and the corresponding depth image to the memory;
a skeleton points calculation module for extracting a 2D human skeleton point information from the 2D color image or the 2D infrared image and correcting the 2D human skeleton point information using the size-depth parameter and the distortion model;
a mapping module for mapping the 2D human skeleton point information to the depth image to obtain a depth information corresponding to the 2D human skeleton point information and combining the corrected 2D human skeleton point information and the depth information to obtain a 3D human skeleton point information; and
a motion identification module for applying a match model to a series of the 3D human skeleton point information in a period of time to identify a motion.
7. The motion identification system of claim 6 , further comprising: an output module for transmitting an alert signal while the motion is identified.
8. The motion identification system of claim 6 , wherein the distortion model is used to correct the distance between the pixel coordinate position of the 2D human skeleton point and the image distortion center.
9. The motion identification system of claim 6 , wherein the match model is a classification model parameter established by a deep learning framework of neural network.
10. The motion identification system of claim 6 , wherein the memory further stores a set of displacement parameters, and the depth image is corrected with the displacement parameters in advance.