Method and device for use in hand gesture recognition

1. A method for use in hand gesture recognition, the method comprising the steps of:

acquiring a depth map of a hand in a current image;

estimating first positions of joints of the hand according to the depth map of the hand;

creating a 3D point cloud of the hand according to the depth map;

matching the first positions of the joints and a stored 3D hand model to the 3D point cloud of the hand to obtain second positions of the joints and first degree of freedom parameters of the joints; and

recognizing the hand's gesture according to the second positions of the joints and the first degree of freedom parameters of the joints,

wherein the step of estimating first positions of joints of the hand according to the depth map comprises: inputting the depth map of the hand to a trained image-hand gesture mapper to obtain the first positions of the joints, wherein the image-hand gesture mapper comprises a mapping relation between hand gestures and the hand depth maps obtained by hand depth map sample training, each of the hand gestures comprising at least one of joint positions and joint degree of freedom parameters.

2. The method of claim 1 , before the step of matching the first position of the joints and a stored 3D hand model to the 3D point cloud of the hand to obtain second positions of the joints and first degree of freedom parameters of the joints, further comprising the steps of:

inputting the depth map of the hand to the image-hand gesture mapper to obtain second degree of freedom parameters of the joints; and

matching the second degree of freedom parameters and the stored 3D hand model to the 3D point cloud of the hand to obtain first shape parameters of the hand, followed by updating the image-hand gesture mapper and the stored 3D hand model according to the first shape parameters of the hand,

wherein the step of inputting the depth map to a trained image-hand gesture mapper to obtain the first positions of the joints comprises inputting the depth map of the hand to the updated image-hand gesture mapper to obtain the first position of the joints,

wherein the step of matching the first positions of the joints and the stored 3D hand model to the 3D point cloud of the hand to obtain second positions of the joints and first degree of freedom parameters of the joints comprises matching the first positions of the joints and the updated 3D hand model to the 3D point cloud of the hand to obtain the second positions of the joints and the first degree of freedom parameters of the joints.

3. The method of claim 2 , wherein the step of inputting the depth map of the hand to the image-hand gesture mapper to obtain second degree of freedom parameters of the joints comprises inputting a first depth map of the hand in a first frame to the image-hand gesture mapper to obtain the second degree of freedom parameters of the joints, wherein, after the step of matching the second degree of freedom parameters of the joints and the stored 3D hand model to the 3D point cloud of the hand to obtain first shape parameters of the hand and then updating the image-hand gesture mapper and the stored 3D hand model according to the first shape parameters of the hand, the method further comprises:

inputting a second depth map of the hand in a second frame to the updated image-hand gesture mapper to obtain third degree of freedom parameters of the joint;

creating a new 3D point cloud of the hand according to the second depth map of the hand in the second frame; and

matching the third degree of freedom parameters and the updated 3D hand model to the new 3D point cloud of the hand to obtain second shape parameters of the hand, followed by updating the image-hand gesture mapper and the stored 3D hand model again according to the second shape parameters of the hand.

4. The method of claim 1 , wherein the image-hand gesture mapper comprises a neural network or a random forest classifier.

5. The method of claim 2 , wherein the first shape parameters comprise at least one of lengths of hand bones and widths of the hand bones.

6. A device for use in hand gesture recognition, the device comprising:

a memory, a processor and a computer program stored in the memory and executable by the processor,

wherein the processor is configured to:

acquire a depth map of a hand in a current image;

estimate first position of joints of the hand according to the depth map of the hand, create a 3D point cloud of the hand according to the depth map of the hand, and match the first position of the joints and a stored 3D hand model to the 3D point cloud of the hand to obtain second positions of the joints and first degree of freedom parameters of the joints;

recognize the hand's gesture according to the second position of the joints and the first degree of freedom parameters of the joints; and

input the depth map of the hand to a trained image-hand gesture mapper to obtain the first positions of the joints, and

wherein the image-hand gesture mapper comprises a mapping relation between hand gestures and hand depth maps obtained by depth map sample training, the hand gestures each comprising at least one of joint positions and joint degree of freedom parameters.

7. The device of claim 6 , wherein the processor is further configured to:

input the depth map of the hand to the image-hand gesture mapper to obtain second degree of freedom parameters of the joints;

match the second degree of freedom parameters of the joints and the stored 3D hand model to the 3D point cloud of the hand to obtain first shape parameters of the hand;

update the image-hand gesture mapper and the stored 3D hand model according to the first shape parameters of the hand; and

input the depth map of the hand to the updated image-hand gesture mapper to obtain the first positions of the joints and match the first positions of the joints and the updated 3D hand model to the 3D point cloud of the hand to obtain the second positions of the joints and the first degree of freedom parameters of the joints.

8. The device of claim 7 , wherein the processor inputs a depth map of the hand in a first frame to the image-hand gesture mapper to obtain second degree of freedom parameters of the joints, inputs a second depth map of the hand in a second frame to the updated image-hand gesture mapper to obtain third degree of freedom parameters of the joints, creates a new 3D point cloud of the hand according to the second depth map of the hand in the second frame, matches the third degree of freedom parameters of the joints and the updated 3D model to the new 3D point cloud of the hand to obtain second shape parameters of the hand, and updates the image-hand gesture mapper and the stored 3D hand model again according to the second shape parameters of the hand.

9. The device of claim 6 , wherein the image-hand gesture mapper comprises a neural network or a random forest classifier.

10. The device of claim 7 , wherein the first shape parameters comprise at least one of lengths of hand bones and widths of the hand bones.

11. A computer-readable storage medium, the computer-readable storage medium storing a computer program for implementing the steps of the method of claim 1 when executed by a processor.