Granted Patent B2

US 12,211,314 · App. 17/610,148 · Granted Jan 28, 2025

Inventors: Honghong Jia (Beijing, CN); Fengshuo Hu (Beijing, CN); Jingru Wang (Beijing, CN)

Assignee: BOE Technology Group Co., Ltd.

G06V40/172G06V10/751G06V10/7715G06V40/20

Abstract

Claims (70)

1. A control method, comprising:

obtaining, by a processor, a first image;

performing, by the processor, face recognition and gesture recognition on the first image;

turning on, by the processor, a gesture control function when a first target face is recognized from the first image and a first target gesture is recognized from the first image; and

returning, by the processor, to the act of obtaining a first image when the first target face is not recognized from the first image or the first target gesture is not recognized from the first image,

wherein performing, by the processor, gesture recognition on the first image comprises:

detecting whether the first image comprises a human body;

segmenting the human body to obtain a plurality of segmented regions when it is detected that the first image comprises the human body, and detecting whether segmented regions comprise an arm region;

detecting whether the arm region comprises a hand region when it is detected that the segmented regions comprise the arm region;

performing gesture recognition on the hand region when it is detected that the arm region comprises the hand region; returning a result that the first target gesture is recognized from the first image when a gesture in the hand region is recognized as the first target gesture; and

returning a result that the first target gesture is not recognized from the first image when it is detected that the first image does not comprise a human body, or that the segmented regions do not comprise an arm region, or that the arm region does not comprise a hand region, or that the gesture in the hand region is not the first target gesture.

2. The method according to claim 1 , further comprising: providing, by the processor, first prompt information on a display interface when the first target face is recognized from the first image and the first target gesture is recognized from the first image, wherein the first prompt information is used for prompting a user that the gesture control function has been turned on.

3. The method according to claim 1 , further comprising: providing, by the processor, second prompt information on a display interface when the first target face is recognized from the first image and the first target gesture is not recognized from the first image, wherein the second prompt information is used for prompting a user to adjust a gesture.

4. The method according to claim 1 , further comprising: providing, by the processor, third prompt information on a display interface when the first target face is not recognized from the first image, wherein the third prompt information is used for prompting a user to adjust an angle of a face facing an acquisition device.

5. The method according to claim 4 , further comprising: providing, by the processor, fourth prompt information on the display interface when the first target face is still not recognized from a first image of another frame re-acquired after the third prompt information is provided, wherein the fourth prompt information is used for prompting that the user has no operation authority.

6. The method according to claim 1 , further comprising: obtaining, by the processor, a second image after the gesture control function is turned on, and performing, by the processor, face recognition and gesture recognition on the second image; and

turning on, by the processor, a function corresponding to a second target gesture when a second target face is recognized from the second image and the second target gesture is recognized from the second image.

7. The method according to claim 6 , further comprising: returning, by the processor, to the act of obtaining the second image when the second target face is not recognized from the second image or the second target gesture is not recognized from the second image; and

turning off, by the processor, the gesture control function when the second target face is not recognized from second images of consecutive multiple frames within a set time period.

8. The method according to claim 1 , wherein the performing, by the processor, face recognition and gesture recognition on the first image comprises:

performing face recognition on the first image; and

performing gesture recognition on the first image after the first target face is recognized from the first image.

9. The method according to claim 1 , wherein the performing, by the processor, face recognition on the first image comprises:

detecting whether the first image comprises a face;

detecting whether the face in the first image is occluded when it is detected that the first image comprises the face;

detecting whether the face in the first image is a front face when it is detected that the face in the first image is not occluded;

performing feature extraction on the first image to obtain face data to be recognized when it is detected that the face in the first image is the front face;

comparing the face data to be recognized with target face data in a face database;

returning a result that the first target face is recognized from the first image when there is target face data matched with the face data to be recognized in the face database; and

returning a result that the first target face is not recognized from the first image when it is detected that the first image does not comprise a face, or that the face in the first image is occluded, or that the face in the first image is not a front face, or that there is no target face data matched with the face data to be recognized in the face database.

10. The method according to claim 9 , further comprising: registering, by the processor, a target face in the face database;

wherein the registering, by the processor, the target face in the face database comprises:

obtaining a registered image;

detecting whether the registered image comprises a face;

returning to the act of obtaining the registered image when it is detected that the registered image does not comprise a face;

detecting whether the face in the registered image is occluded when it is detected that the registered image comprises the face;

returning to the act of obtaining the registered image when it is detected that the face in the registered image is occluded;

detecting whether the face in the registered image is a front face when it is detected that the face in the registered image is not occluded;

returning to the act of obtaining the registered image when it is detected that the face in the registered image is not a front face;

performing feature extraction on the registered image to obtain face data to be registered when it is detected that the face in the registered image is a front face;

comparing the face data to be registered with registered face data in the face database;

providing fifth prompt information on a display interface when there is registered face data matched with the face data to be registered in the face database, wherein the fifth prompt information is used for prompting that a user is already registered; and

assigning an identifier to the face data to be registered when there is no registered face data matched with the face data to be registered in the face database, and saving the face data to be registered in the face database.

11. The method according to claim 1 , wherein the first target gesture comprises an OK gesture.

12. The method according to claim 6 , wherein the turning on, by the processor, the function corresponding to the second target gesture comprises:

determining a mapping position of a palm of one hand on a display interface when the second target gesture is the palm of one hand, and selecting an icon corresponding to the mapping position; and

turning on a function indicated by the icon corresponding to the mapping position after the palm of one hand is detected and when it is detected that the second target gesture is a fist of one hand.

13. An electronic device, comprising the processor of claim 1 , a memory, and a display; the display is connected to the processor and is adapted to provide a display interface; the memory is adapted to store a computer program, and when the computer program is executed by the processor, acts of the control method according to claim 1 are implemented.

14. A non-transitory computer-readable storage medium, storing a computer program, wherein when the computer program is executed by the processor of claim 1 , acts of the control method according to claim 1 are implemented.

15. The method according to claim 2 , further comprising: obtaining, by the processor, a second image after the gesture control function is turned on, and performing, by the processor, face recognition and gesture recognition on the second image; and

turning on, by the processor, a function corresponding to a second target gesture when a second target face is recognized from the second image and the second target gesture is recognized from the second image.

16. The method according to claim 15 , further comprising: returning, by the processor, to the act of obtaining a second image when the second target face is not recognized from the second image or the second target gesture is not recognized from the second image; and

turning off, by the processor, the gesture control function when the second target face is not recognized from second images of consecutive multiple frames within a set time period.

17. The method according to claim 8 , wherein the performing, by the processor, face recognition on the first image comprises:

detecting whether the first image comprises a face;

detecting whether the face in the first image is occluded when it is detected that the first image comprises the face;

detecting whether the face in the first image is a front face when it is detected that the face in the first image is not occluded;

performing feature extraction on the first image to obtain face data to be recognized when it is detected that the face in the first image is the front face;

comparing the face data to be recognized with target face data in a face database;

returning a result that the first target face is recognized from the first image when there is target face data matched with the face data to be recognized in the face database; and

returning a result that the first target face is not recognized from the first image when it is detected that the first image does not comprise a face, or that the face in the first image is occluded, or that the face in the first image is not a front face, or that there is no target face data matched with the face data to be recognized in the face database.

18. The method according to claim 8 , wherein the performing, by the processor, gesture recognition on the first image comprises:

detecting whether the first image comprises a human body;

segmenting the human body to obtain a plurality of segmented regions when it is detected that the first image comprises the human body, and detecting whether segmented regions comprise an arm region;

detecting whether the arm region comprises a hand region when it is detected that the segmented regions comprise the arm region;

performing gesture recognition on the hand region when it is detected that the arm region comprises the hand region; returning a result that the first target gesture is recognized from the first image when a gesture in the hand region is recognized as the first target gesture; and

returning a result that the first target gesture is not recognized from the first image when it is detected that the first image does not comprise a human body, or that the segmented regions do not comprise an arm region, or that the arm region does not comprise a hand region, or that the gesture in the hand region is not the first target gesture.

19. The method according to claim 7 , wherein the turning on, by the processor, the function corresponding to the second target gesture comprises:

determining a mapping position of a palm of one hand on a display interface when the second target gesture is the palm of one hand, and selecting an icon corresponding to the mapping position; and

turning on a function indicated by the icon corresponding to the mapping position after the palm of one hand is detected and when it is detected that the second target gesture is a fist of one hand.

Assignments (1)

Continuity (1)

References Cited (38)

Cited By (2)

US 12,493,353 US 12,495,982