Granted Patent B2

US 12,558,613 · App. 18/258,712 · Granted Feb 24, 2026

Inventors: Hang Liu (Shenzhen, CN); Xiaohan Chen (Shenzhen, CN)

Assignee: HUAWEI TECHNOLOGIES CO., LTD.

A63F13/22A63F13/213A63F13/23A63F13/67

Abstract

Claims (99)

1 . A method, applied to an electronic device, wherein the method comprises:

obtaining, by a camera, a first image comprising a first user action, wherein the first user action is a first version of the first user action;

obtaining, by the camera, a second image comprising a second user action, wherein the second user action is a second version of the first user action;

obtaining a first action template for recognizing somatosensory actions performed by a user;

obtaining, by the camera, a third image comprising a third user action;

determining whether the third image matches the first user action;

displaying, when the third image does not match the first user action, a prompt window that requests input of a new second image or a new third image;

associating a first key with the first action template;

performing a first operation on the first key;

generating, when the first operation is performed, a first key signal for controlling the electronic device to perform a first action;

obtaining, by the camera, an image sequence comprising a user action;

generating, when recognizing based on the first action template that the user action is a first somatosensory action, a second key signal; and

performing, by controlling execution of a game application on the electronic device, the first action in response to the second key signal.

2 . The method of claim 1 , wherein after associating the first key signal with the first action template, the method further comprises displaying a first interface, and wherein the first interface displays an association relationship between the first action template and the first key.

3 . The method of claim 1 , wherein before obtaining the first action template, the method further comprises:

recognizing skeleton nodes in the first image to obtain a first set of skeleton node data;

recognizing skeleton nodes in the second image to obtain a second set of skeleton node data;

calculating a difference between the first set of skeleton node data and the second set of skeleton node data to obtain a threshold interval; and

generating the first action template comprising the threshold interval and comprising at least one of the first set of skeleton node data or the second set of skeleton node data.

4 . The method of claim 3 , wherein obtaining the first image comprising the user action comprises:

collecting, by the camera, a first image sequence in which the user completes a somatosensory action; and

determining, as the first image, an image frame having a smallest change amplitude of a somatosensory action of the user compared with a previous image frame in the first image sequence, and

wherein obtaining the second image comprising the somatosensory action of the user comprises:

collecting a second image sequence in which the user completes a somatosensory action; and

determining, as the second image, an image frame having a smallest change amplitude of a somatosensory action of the user compared with a previous image frame in the second image sequence.

5 . The method of claim 3 , wherein after generating the first action template, the method further comprises:

recognizing skeleton nodes in the third image to obtain a third set of skeleton node data;

determining that the third image matches the first user action when the third set of skeleton node data matches the first action template; and

locking the first action template.

6 . The method of claim 5 , wherein determining that the third set of skeleton node data matches the first action template further comprises:

calculating a difference between the third set of skeleton node data and the first set of skeleton node data; and

determining, when the difference falls within the threshold interval, that the third set of skeleton node data matches the first action template.

7 . The method of claim 3 , wherein after obtaining the image sequence comprising the user action, the method further comprises:

obtaining, by the camera, a fourth image from the image sequence;

recognizing skeleton nodes in the fourth image to obtain a fourth set of skeleton node data;

calculating a difference between the fourth set of skeleton node data and the first set of skeleton node data; and

recognizing, when the difference falls within the threshold interval, that the user action indicated by the image sequence matches the first action template.

8 . The method of claim 1 , wherein the method further comprises storing the first action template as a preset action template.

9 . The method of claim 1 , wherein obtaining the first action template further comprises:

displaying a second interface, wherein the second interface displays a plurality of preset action templates for selection; and

selecting the first action template from the plurality of preset action templates.

10 . The method of claim 9 , wherein the method further comprises:

learning a plurality of user actions of the user, and separately storing the plurality of user actions as the plurality of preset action templates; or

obtaining a plurality of shared preset action templates.

11 . The method of claim 9 , wherein selecting the first action template from the plurality of preset action templates further comprises selecting, when recognizing that an action performed by the user matches the first action template, the first action template from the plurality of preset action templates.

12 . An electronic device, comprising:

one or more memories configured to store computer instructions; and

one or more processors coupled to the one or more memories and configured to execute the computer instructions to cause the electronic device to:

obtain, by a camera, a first image comprising a first user action, wherein the first user action is a first version of the first user action;

obtain, by the camera, a second image comprising a second user action, wherein the second user action is a second version of the first user action;

obtain a first action template for recognizing somatosensory actions performed by a user;

obtain, by the camera, a third image comprising a third user action;

determine whether the third image matches the first user action;

display, when the third image does not match the first user action, a prompt window that requests input of a new second image or a new third image;

associate a first key with the first action template;

perform a first operation on the first key;

generate, when the first operation is performed, a first key signal for controlling the electronic device to perform a first action;

obtain, by the camera, an image sequence comprising a user action;

generate, when recognizing based on the first action template that the user action is a first somatosensory action, a second key signal; and

perform, by controlling execution of a game application on the electronic device, the first action in response to the second key signal.

13 . The electronic device of claim 12 , wherein the one or more processors are further configured to execute the computer instructions to cause the electronic device to display a first interface, wherein the first interface displays an association relationship between the first action template and the first key.

14 . The electronic device of claim 12 , wherein the one or more processors are further configured to execute the computer instructions to cause the electronic device to:

recognize skeleton nodes in the first image to obtain a first set of skeleton node data;

recognize skeleton nodes in the second image to obtain a second set of skeleton node data;

calculate a difference between the first set of skeleton node data and the second set of skeleton node data to obtain a threshold interval; and

generate the first action template, wherein the first action template comprises the first set of skeleton node data and/or the second set of skeleton node data, and the threshold interval.

15 . The electronic device of claim 14 , wherein the one or more processors are further configured to execute the computer instructions to cause the electronic device to:

collect, by the camera, a first image sequence in which the user completes a somatosensory action; and

determine, as the first image, an image frame having a smallest change amplitude of a somatosensory action of the user compared with a previous image frame in the first image sequence;

collect a second image sequence in which the user completes a somatosensory action; and

determine, as the second image, an image frame having a smallest change amplitude of a somatosensory action of the user compared with a previous image frame in the second image sequence.

16 . The electronic device of claim 14 , wherein the one or more processors are further configured to execute the computer instructions to cause the electronic device to:

recognize skeleton nodes in the third image to obtain a third set of skeleton node data;

determine that the third image matches the first user action when the third set of skeleton node data matches the first action template; and

lock the first action template.

17 . The electronic device of claim 16 , wherein the one or more processors are further configured to execute the computer instructions to cause the electronic device to:

calculate a difference between the third set of skeleton node data and the first set of skeleton node data; and

determine, when the difference falls within the threshold interval, that the third set of skeleton node data matches the first action template.

18 . The electronic device of claim 12 , wherein the one or more processors are further configured to execute the computer instructions to cause the electronic device to:

display a second interface, wherein the second interface displays a plurality of preset action templates for selection; and

select the first action template from the plurality of preset action templates.

19 . The electronic device of claim 14 , wherein the one or more processors are further configured to execute the computer instructions to cause the electronic device to:

obtain a fourth image from the image sequence;

recognize skeleton nodes in the fourth image to obtain a fourth set of skeleton node data;

calculate a difference between the fourth set of skeleton node data and the first set of skeleton node data; and

recognize, when the difference falls within the threshold interval, that the user action indicated by the image sequence matches the first action template.

20 . A non-transitory computer-readable storage medium comprising instructions, wherein when the instructions are executed on an electronic device, the instructions cause the electronic device to:

obtain, by a camera, a first image comprising a first user action, wherein the first user action is a first version of the first user action;

obtain, by the camera, a second image comprising a second user action, wherein the second user action is a second version of the first user action;

obtain a first action template for recognizing somatosensory actions performed by a user;

obtain, by the camera, a third image comprising a third user action;

determine whether the third image matches the first user action;

display, when the third image does not match the first user action, a prompt window that requests input of a new second image or a new third image;

associate a first key with the first action template;

perform a first operation on the first key;

generate, when the first operation is performed, a first key signal for controlling the electronic device to perform a first action;

obtain, by the camera, an image sequence comprising a user action;

generate, when recognizing based on the first action template that the user action is a first somatosensory action, a second key signal; and

perform, by controlling execution of a game application on the electronic device, the first action in response to the second key signal.

Assignments (2)

Priority Claims (2)

Continuity (1)

References Cited (19)