Method, device and readable storage medium for processing control instruction based on gesture recognition
View Patent ↗The present disclosure provides a method, a device and a readable storage medium for processing a control instruction based on gesture recognition, the method including: obtaining a trigger-gesture-recognition instruction; capturing a set of to-be-recognized images corresponding to a to-be-recognized user according to the trigger-gesture-recognition instruction, where the set of to-be-recognized images includes to-be-recognized images of multiple frames; determining location information of a hand of the to-be-recognized user respectively in each frame of the to-be-recognized images via a preset body recognizing model; and according to the location information, obtaining gesture information corresponding to the hand via a preset gesture recognizing model; and according to the gesture information corresponding to the hand, obtaining a control instruction corresponding to the gesture information. Thus, the amount of computation for gesture recognition is reduced, and efficiency of gesture recognition is improved, enabling a remote gesture control of an intelligent terminal.
1. A method for processing a control instruction based on gesture recognition, applied in an intelligent terminal, wherein the method comprises:
obtaining a trigger-gesture-recognition instruction;
capturing a set of to-be-recognized images corresponding to a to-be-recognized user according to the trigger-gesture-recognition instruction, wherein the set of to-be-recognized images includes to-be-recognized images of multiple frames;
determining location information of a hand of the to-be-recognized user respectively in each frame of the to-be-recognized images via a preset body recognizing model; and according to the location information of the hand of the to-be-recognized user in each frame of the to-be-recognized images, obtaining gesture information corresponding to the hand via a preset gesture recognizing model; and
obtaining, according to the gesture information corresponding to the hand, the control instruction corresponding to the gesture information;
wherein the determining location information of the hand of the to-be-recognized user respectively in each frame of the to-be-recognized images via the preset body recognizing model comprises:
based on an earlier-to-later time order of capturing the to-be-recognized images, determining the location information of the hand of the to-be-recognized user respectively in first N frames of the to-be-recognized images via the preset body recognizing model;
according to the location information of the hand of the to-be-recognized user in the first N frames of the to-be-recognized images, obtaining a hand recognition area; and
according to the hand recognition area, determining the location information of the hand of the to-be-recognized user respectively in to-be-recognized images other than the first N frames in the set of to-be-recognized images,
wherein N is a positive integer greater than or equal to 1.
2. The method according to claim 1 , wherein the obtaining, according to the gesture information corresponding to the hand, the control instruction corresponding to the gesture information comprises:
if multiple candidate control instructions corresponding to the gesture information are obtained, according to a track and a direction of a movement of the hand from the gesture information, predicting to-be-completed gesture information corresponding to the hand; and
according to the gesture information and the to-be-completed gesture information, determining the control instruction from the candidate control instructions.
3. The method according to claim 1 , further comprising:
obtaining hand recognition sample data, and building a body recognition training model; and
according to the hand recognition sample data, training the body recognition training model to obtain the body recognizing model.
4. The method according to claim 1 , further comprising:
obtaining gesture recognition sample data, and building a gesture recognition training model; and
according to the gesture recognition sample data, training the gesture recognition training model to obtain the gesture recognizing model.
5. A device for processing a control instruction based on gesture recognition, applied in an intelligent terminal, wherein the device comprises: a memory and a processor, wherein
the memory is configured to store an instruction executable by the processor; and
the processor is configured to perform the method according to claim 1 .
6. A non-transitory computer readable storage medium, storing a computer-executed instruction which, when executed by a processor, implements the method according to claim 1 .
7. A device for processing a control instruction based on gesture recognition, applied in an intelligent terminal, wherein the device comprises: a processor and a non-transitory computer-readable medium for storing program codes, which, when executed by the processor, cause the processor to:
obtain a trigger-gesture-recognition instruction;
capture a set of to-be-recognized images corresponding to a to-be-recognized user according to the trigger-gesture-recognition instruction, wherein the set of to-be-recognized images includes to-be-recognized images of multiple frames;
determine location information of a hand of the to-be-recognized user respectively in each frame of the to-be-recognized images via a preset body recognizing model and obtain gesture information corresponding to the hand via a preset gesture recognizing model according to the location information of the hand of the to-be-recognized user in each frame of the to-be-recognized images; and
obtain, according to the gesture information corresponding to the hand, the control instruction corresponding to the gesture information;
wherein the program codes further cause the processor to:
determine the location information of the hand of the to-be-recognized user respectively in first N frames of the to-be-recognized images via the preset body recognizing model based on an earlier-to-later time order of capturing the to-be-recognized images, wherein N is a positive integer greater than or equal to 1;
obtain a hand recognition area according to the location information of the hand of the to-be-recognized user in the first N frames of the to-be-recognized images; and
determine the location information of the hand of the to-be-recognized user respectively in to-be-recognized images other than the first N frames in the set of to-be-recognized images according to the hand recognition area.
8. The device according to claim 7 , wherein the program codes further cause the processor to:
predict to-be-completed gesture information corresponding to the hand according to a track and a direction of a movement of the hand from the gesture information if multiple candidate control instructions corresponding to the gesture information are obtained; and
determine the control instruction from the candidate control instructions according to the gesture information and the to-be-completed gesture information.
9. The device according to claim 7 , wherein the program codes further cause the processor to:
obtain hand recognition sample data, and build a body recognition training model; and
train, according to the hand recognition sample data, the body recognition training model to obtain the body recognizing model.
10. The device according to claim 7 , wherein the program codes further cause the processor to:
obtain gesture recognition sample data, and build a gesture recognition training model; and
train, according to the gesture recognition sample data, the gesture recognition training model to obtain the gesture recognizing model.