Methods and apparatuses for recognizing gesture, electronic devices and storage media

1. A method of recognizing a gesture, comprising:

detecting at least one hand region from a video image and obtaining hand image information of each of the at least one hand region;

obtaining hand motion information of each of the at least one hand region by tracking the at least one hand region;

determining a gesture corresponding to each of the at least one hand region according to the hand image information and/or the hand motion information of each of the at least one hand region; wherein the gesture comprises at least one of a single-hand static gesture, a single-hand dynamic gesture, a double-hand static gesture or a double-hand dynamic gesture;

wherein detecting the at least one hand region from the video image comprises:

determining that there is only one hand region by detecting the video image; and

determining the gesture corresponding to each of the at least one hand region according to the hand image information and/or the hand motion information of each of the at least one hand region comprises:

obtaining a first recognition result by performing a single-hand static gesture recognition for the hand image information of the hand region; and

in response to that the first recognition result is yes, determining that the gesture corresponding to the hand region is the single-hand static gesture;

wherein determining the gesture corresponding to each of the at least one hand region according to the hand image information and/or the hand motion information of each of the at least one hand region comprises:

in response to that the first recognition result is no, obtaining a second recognition result by performing a first single-hand dynamic gesture recognition based on the hand motion information of the hand region; and

in response to that the second recognition result is yes, determining that the gesture corresponding to the hand region is the single-hand dynamic gesture;

wherein determining the gesture corresponding to each of the at least one hand region according to the hand image information and/or the hand motion information of each of the at least one hand region comprises:

in response to that the second recognition result is no, obtaining a third recognition result by performing a second single-hand dynamic gesture recognition based on the hand image information and the hand motion information of the hand region; and

in response to that the third recognition result is yes, determining that the gesture corresponding to the hand region is the single-hand dynamic gesture.

2. The method according to claim 1 , wherein obtaining the hand motion information of each of the at least one hand region by tracking the at least one hand region comprises:

obtaining one or more prediction results by predicting the hand motion information of each of the at least one hand region based on a prediction model; wherein the hand motion information comprises a hand position and a hand motion speed;

matching the one or more prediction results with one or more detection results of the hand motion information of the at least one hand region in a current frame of video image;

updating one or more parameters of the prediction model using one or more of the detection results matched with the one or more prediction results;

tracking each of the at least one hand region; and

obtaining the hand motion information of each of the at least one hand region by obtaining the hand motion information of a tracked same hand region.

3. An electronic device, comprising:

a processor;

a memory, in communication connection with the processor;

at least one program, stored in the memory and configured to be executed by the processor; wherein the at least one program is executed by the processor to implement the method according to claim 1 .

4. The electronic device according to claim 3 , wherein when obtaining the hand motion information of each of the at least one hand region by tracking the at least one hand region, the processor is configured to:

obtain one or more prediction results by predicting the hand motion information of each of the at least one hand region based on a prediction model; wherein the hand motion information comprises a hand position and a hand motion speed;

match the one or more prediction results with one or more detection results of the hand motion information of the at least one hand region in a current frame of video image;

update one or more parameters of the prediction model using one or more of the detection results matched with the one or more prediction results;

track each of the at least one hand region; and

obtain the hand motion information of each of the at least one hand region by obtaining the hand motion information of a tracked same hand region.

5. A non-transitory computer readable storage medium, storing computer instructions, wherein the computer instructions are run on a computer to implement the method according to claim 1 .

6. A method of recognizing a gesture, comprising:

detecting at least one hand region from a video image and obtaining hand image information of each of the at least one hand region;

obtaining hand motion information of each of the at least one hand region by tracking the at least one hand region;

determining a gesture corresponding to each of the at least one hand region according to the hand image information and/or the hand motion information of each of the at least one hand region; wherein the gesture comprises at least one of a single-hand static gesture, a single-hand dynamic gesture, a double-hand static gesture or a double-hand dynamic gesture, wherein

detecting the at least one hand region from the video image comprises:

determining that there are at least two hand regions by detecting the video image; and

determining the gesture corresponding to each of the at least one hand region according to the hand image information and/or the hand motion information of each of the at least one hand region comprises:

for any two hand regions, determining a confidence level of the two the hand regions belonging to two hands of a same person according to the hand image information and the hand motion information of the two hand regions;

in response to that one or more confidence levels are greater than a preset threshold, determining that the two hand regions with the highest confidence level belong to two hands of a same person;

obtaining a fourth recognition result by performing a double-hand static gesture recognition for the hand image information of the two hand regions belonging to two hands of a same person; and

in response to that the fourth recognition result is yes, determining that the gesture corresponding to the two hand regions is the double-hand static gesture.

7. The method according to claim 6 , wherein determining the gesture corresponding to each of the at least one hand region according to the hand image information and/or the hand motion information of each of the at least one hand region comprises:

in response to the fourth recognition result is no, obtaining a fifth recognition result by performing a first double-hand dynamic gesture recognition based on the hand motion information of the two hand regions; and

in response to that the fifth recognition result is yes, determining that the gesture corresponding to the two hand regions is the double-hand dynamic gesture.

8. The method according to claim 7 , wherein determining the gesture corresponding to each of the at least one hand region according to the hand image information and/or the hand motion information of each of the at least one hand region comprises:

in response to that the fifth recognition result is no, obtaining a sixth recognition result by performing a second double-hand dynamic gesture recognition based on the hand image information and the hand motion information of the two hand regions;

in response to that the sixth recognition result is yes, determining that the gesture corresponding to the two hand regions is the double-hand dynamic gesture.

9. The method according to claim 6 , wherein for any two hand regions, determining the confidence level that the two hand regions belonging to two hands of a same person according to the hand image information and the hand motion information of the two hand regions comprises:

in response to that no confidence level is greater than the preset threshold, separately performing a single-hand gesture recognition for each of the at least one hand region; wherein the single-hand gesture recognition comprises at least one of a single-hand static gesture recognition, a first single-hand dynamic gesture recognition, or a second single-hand dynamic gesture recognition.

10. The method according to claim 6 , wherein obtaining the hand motion information of each of the at least one hand region by tracking the at least one hand region comprises:

obtaining one or more prediction results by predicting the hand motion information of each of the at least one hand region based on a prediction model; wherein the hand motion information comprises a hand position and a hand motion speed;

matching the one or more prediction results with one or more detection results of the hand motion information of the at least one hand region in a current frame of video image;

updating one or more parameters of the prediction model using one or more of the detection results matched with the one or more prediction results;

tracking each of the at least one hand region; and

obtaining the hand motion information of each of the at least one hand region by obtaining the hand motion information of a tracked same hand region.

11. An electronic device, comprising:

a processor;

a memory, in communication connection with the processor;

at least one program, stored in the memory and configured to be executed by the processor; wherein the at least one program is configured to:

detect at least one hand region from a video image and obtain hand image information of each of the at least one hand region;

obtain hand motion information of each of the at least one hand region by tracking the at least one hand region;

determine a gesture corresponding to each of the at least one hand region according to the hand image information and/or the hand motion information of each of the at least one hand region; wherein the gesture comprises at least one of a single-hand static gesture, a single-hand dynamic gesture, a double-hand static gesture, or a double-hand dynamic gesture, wherein

when detecting the at least one hand region from the video image, the processor is configured to:

determine that there are at least two hand regions by detecting the video image; and

when determining the gesture corresponding to each of the at least one hand region according to the hand image information and/or the hand motion information of each of the at least one hand region, the processor is configured to:

for any two hand regions, determine a confidence level of the two hand regions belonging to two hands of a same person according to the hand image information and the hand motion information of the two hand regions;

in response to that one or more confidence levels are greater than a preset threshold, determine that the two hand regions with the highest confidence level belong to two hands of a same person;

obtain a fourth recognition result by performing a double-hand static gesture recognition for the hand image information of the two hand regions belonging to two hands of a same person;

in response to that the fourth recognition result is yes, determine that the gesture corresponding to the two hand regions is the double-hand static gesture.

12. The electronic device according to claim 11 , wherein when obtaining the hand motion information of each of the at least one hand region by tracking the at least one hand region, the processor is configured to:

obtain one or more prediction results by predicting the hand motion information of each of the at least one hand region based on a prediction model; wherein the hand motion information comprises a hand position and a hand motion speed;

match the one or more prediction results with one or more detection results of the hand motion information of the at least one hand region in a current frame of video image;

update one or more parameters of the prediction model using one or more of the detection results matched with the one or more prediction results;

track each of the at least one hand region; and

obtain the hand motion information of the at least one hand region by obtaining the hand motion information of a tracked same hand region.

13. The electronic device according to claim 12 , wherein when determining the gesture corresponding to each of the at least one hand region according to the hand image information and/or the hand motion information of each of the at least one hand region, the processor is configured to:

in response to the fourth recognition result is no, obtain a fifth recognition result by performing a first double-hand dynamic gesture recognition based on the hand motion information of the two hand regions;

in response to that the fifth recognition result is yes, determine that the gesture corresponding to the two hand regions is the double-hand dynamic gesture.

14. The electronic device according to claim 13 , wherein when determining the gesture corresponding to each of the at least one hand region according to the hand image information and/or the hand motion information of each of the at least one hand region, the processor is configured to:

in response to that the fifth recognition result is no, obtain a sixth recognition result by performing a second double-hand dynamic gesture recognition based on the hand image information and the hand motion information of the two hand regions;

in response to that the sixth recognition result is yes, determine that the gesture corresponding to the two hand regions is the double-hand dynamic gesture.

15. The electronic device according to claim 12 , wherein when determining, for any two hand regions, the confidence level of the two hand regions belonging to two hands of a same person according to the hand image information and the hand motion information of the two hand regions, the processor is configured to:

in response to that no confidence level is greater than the preset threshold, separately perform a single-hand gesture recognition for each of the at least one hand region; wherein the single-hand gesture recognition comprises at least one of a single-hand static gesture recognition, a first single-hand dynamic gesture recognition, or a second single-hand dynamic gesture recognition.

16. A non-transitory computer readable storage medium, storing computer instructions, wherein the computer instructions are run on a computer to implement the method according to claim 6 .