IP Library › Granted Patent US 11,921,931
Granted Patent B2
US 11,921,931 · App. 17/125,462 · Granted Mar 5, 2024

Methods and systems for multi-precision discrete control of a user interface control element of a gesture-controlled device

Inventors: Wei Zhou (Richmond Hill, CA); Sachi Mizobuchi (Toronto, CA); Rafael Veras Guimaraes (North York, CA); Ghazaleh Saniee-Monfared (Toronto, CA); Wei Li (Richmond Hill, CA)
Assignee: HUAWEI TECHNOLOGIES CO., LTD.
G06F3/017G06F3/04815G06F3/04845G06F3/04847G06V20/46G06V40/107G06V40/28
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,921,931
App. No.
17/125,462
Granted
Mar 5, 2024
Kind
B2
Abstract

Methods and systems for discrete control of user interface elements such as graphical widgets are described. A first plurality of video frames are captured and processed to recognize hand gestures therein. A dragging mode is activated in response to a dragging mode activation hand gesture in the first plurality of frames. A second plurality of video frames are captured and processed to allow recognition of hand gestures therein. Dragging, an element of a user interface control, in response to recognition of a discrete control hand gesture in the second plurality of frames. The discrete control of a user interface control using mid-air hand gestures allows precise setting of system parameters associated with the user interface control.

Claims (51)

1. A method for controlling a user interface control element displayed on a display of a gesture-control device, the user interface control element the method comprising:

obtaining a first plurality of video frames of a video captured by an image-capturing device of the gesture-controlled device;

activating a dragging mode for the user interface control element in response to recognition of a dragging mode activation mid-air hand gesture performed by a user of the gesture-controlled device in the first plurality of video frames;

obtaining a second plurality of video frames of the video captured by the image-capturing device of the gesture-controlled device;

setting a default dragging direction for a draggable graphical widget of the user interface control element in response to recognition of a direction setting mid-air hand gesture in the second plurality of video frames; and

moving, the draggable graphical widget of the user interface control element, by a predefined distance, in the default dragging direction in response to recognition of a dynamic discrete control mid-air hand gesture performed by the user of the gesture-controlled device in the second plurality of video frames.

2. The method of claim 1 , further comprising adjusting a system parameter associated with the user interface control element by a predefined amount.

3. The method of claim 1 , further comprising deactivating the dragging mode for the user interface control element in response to recognition of a dragging mode deactivation hand gesture performed by the user of the gesture-controlled device in the second plurality of video frames.

4. The method of claim 1 , wherein recognition of the direction setting mid-air hand gesture comprises recognition of a dynamic direction setting mid-air hand gesture by a primary hand in the second plurality of video frames of the video.

5. The method of claim 4 , wherein recognition of the dynamic direction setting mid-air hand gesture comprises determining an initial position of the primary hand of the user within the first plurality of video frames; and

recognition of the dynamic direction setting mid-air hand gesture comprises determining that the primary hand has moved from the initial position to a subsequent position within the second plurality of video frames.

6. The method of claim 5 , wherein:

the initial position is determined between a point on the primary hand and a point of reference within the first plurality of video frames; and

the subsequent position is determined between a point on the primary hand and a point of reference within the second plurality of video frames.

7. The method of claim 6 , wherein:

the initial position is determined by an initial horizontal distance between the point on the primary hand and the point of reference within the first plurality of video frames; and

the subsequent position is determined by a subsequent horizontal distance between the point on the primary hand and the point of reference within the second plurality of video frames.

8. The method of claim 6 , wherein:

the initial position is determined by an initial height between the point on the primary hand and the point of reference within the first plurality of video frames; and

the subsequent position is determined by a subsequent height between the point on the primary hand and the point of reference within the second plurality of video frames.

9. The method of claim 5 , wherein setting the default dragging direction further comprises:

comparing the initial position and the subsequent position; and

determining whether a difference between the initial position and the subsequent position is greater than a particular threshold.

10. The method of claim 9 , wherein the predefined distance is based on the difference between the initial position and the subsequent position.

11. The method of claim 1 , wherein recognition of the direction setting mid-air hand gesture comprises recognition of a direction setting mid-air hand gesture by a secondary hand in the second plurality of video frames.

12. The method of claim 1 , wherein the dynamic discrete control mid-air hand gesture comprises a dynamic pinching mid-air hand gesture.

13. The method of claim 12 , wherein the dynamic pinching mid-air hand gesture comprises a pinch open hand gesture followed by a pinch closed hand gesture.

14. The method of claim 1 , wherein:

the user interface control element comprises a slider control; and

moving the draggable graphical widget of the user interface control element comprises dragging a slider element along a track of the slider control.

15. The method of claim 1 , wherein moving the draggable graphical widget of the user interface control element by a predefined distance, in the default dragging direction comprises:

incrementing or decrementing the draggable graphical widget by a discrete amount along a continuous control element, based on the default dragging direction, in response to recognition of the dynamic discrete control mid-air hand gesture performed by the user in the second plurality of video frames.

16. A gesture-controlled device comprising:

an image-capturing device;

a processor;

a memory coupled to the processor, the memory storing machine-executable instructions which, when executed by the processor, cause the gesture-controlled device to:

process a first plurality of video frames of a video captured by the image-capturing device to recognize hand gestures performed by a user of the gesture-controlled device;

activate a dragging mode for controlling a user interface control element in response to recognition of a dragging mode activation mid-air hand gesture in the first plurality of video frames;

process a second plurality of video frames of the video captured by the image-capturing device to recognize hand gestures performed by the user of the gesture-controlled device;

set a default dragging direction for a draggable graphical widget of the user interface control element in response to recognition of a direction setting mid-air hand gesture in the second plurality of video frames; and

move the draggable graphical widget of the user interface control element, by a predefined distance, in the default dragging direction in response to recognition of a dynamic discrete control mid-air hand gesture performed by the user in the second plurality of video frames.

17. The gesture-controlled device of claim 16 , wherein the machine-executable instructions further cause the gesture-controlled device to deactivate the dragging mode for the user interface control element in response to recognition of a dragging mode deactivation hand gesture performed by the user of the gesture-controlled device in the second plurality of video frames.

18. The gesture-controlled device of claim 8 , wherein the recognition of the direction setting mid-air hand gesture comprises recognition of a dynamic direction setting mid-air hand gesture by a primary hand in the second plurality of video frames of the video.

19. The gesture-controlled device of claim 16 , wherein the machine-executable instructions cause the gesture-controlled device to move the draggable graphical widget of the user interface control element by a predefined distance, in the default dragging direction by:

incrementing or decrementing the draggable graphical widget by a discrete amount along a continuous control element, based on the default dragging direction, in response to recognition of the dynamic discrete control mid-air hand gesture performed by the user in the second plurality of video frames.

20. A non-transitory computer-readable medium having machine-executable instructions stored thereon, the machine-executable instructions, when executed by a processor of a gesture-controlled device, cause the gesture-controlled device to:

process a first plurality of video frames of a video captured by an image-capturing device to recognize hand gestures performed by a user of the gesture-controlled device;

activate a dragging mode for controlling a user interface control element in response to recognition of a dragging mode activation mid-air hand gesture in the first plurality of video frames;

process a second plurality of video frames of the video captured by the image-capturing device to recognize hand gestures performed by the user of the gesture-controlled device;

set a default dragging direction for a draggable graphical widget of the user interface control element in response to recognition of a direction setting mid-air hand gesture in the second plurality of video frames; and

move the draggable graphical widget of the user interface control element, by a predefined distance, in the default dragging direction in response to recognition of a dynamic discrete control mid-air hand gesture performed by the user in the second plurality of video frames.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 31, 2022
From: ZHOU, WEI; MIZOBUCHI, SACHI; VERAS GUIMARAES, RAFAEL; SANIEE-MONFARED, GHAZALEH; LI, WEI
To: HUAWEI TECHNOLOGIES CO., LTD.
Reel/Frame 059455/0961 →
Continuity (1)
Related Publication 20220197392A1 · Jun 23, 2022