Method and device for adjusting the control-display gain of a gesture controlled electronic device
Methods, devices, and processor-readable media for adjusting the control-display gain of a gesture-controlled device are described. Adjusting the control-display gain may facilitate user interaction with content or UI elements rendered on a display screen of the gesture-controlled device. The control-display gain may be adjusted based on a property of how a mid-air dragging gesture is being performed by a user's hand. The property may be the location of the gesture, the orientation of the hand performing the gesture, or the velocity of the gesture. A hand that becomes stationary for a threshold time period while performing the dragging gesture may adjust the control-display gain to a different level. Control-display gain may be set to a different value based on the current velocity of the hand performing the gesture. The control-display gain levels may be selected from a continuous range of values or a set of discrete values. Devices for performing the methods are described.
1. A method for adjusting the control-display gain of a gesture-controlled device, the method comprising:
processing a video frame to detect a location and shape of a hand in the video frame;
generating gesture data for the video frame based on the location and shape of the hand;
determining that the hand is performing a dragging hand gesture based on gesture data for the video frame and gesture data for one or more previous video frames;
identifying a property of the hand performing the dragging hand gesture; and
adjusting a control-display gain for the gesture-controlled device to a control-display gain determined based on the identified property.
2. The method of claim 1 , wherein the property is the location of the hand.
3. The method of claim 2 , the method further comprising:
storing the location of the hand in a queue that includes one or more locations of the hand while the gesture-controlled device is in a dragging state;
determining a movement of the hand with respect to a first axis based on the one or more locations of the hand stored in the queue; and
applying the control-display gain to the movement to generate a drag control output value,
wherein the control-display gain is determined based on the location of the hand with respect to a second axis orthogonal to the first axis.
4. The method of claim 2 , wherein:
identifying the location of the hand comprises determining that the hand is located within a first region of the video frame; and
the control-display gain is a first control-display gain corresponding to the first region.
5. The method of claim 1 , wherein the property is an orientation of the hand relative to a reference orientation.
6. The method of claim 5 , wherein the orientation of the hand comprises an angle of the hand relative to the reference orientation.
7. The method of claim 6 , wherein the angle of the hand is measured with respect to a vertical plane.
8. The method of claim 6 , wherein:
identifying the orientation of the hand comprises determining that the angle of the hand is within a first angle range defined between a first angle and a second angle; and
adjusting the control-display gain comprises adjusting the control-display gain to a first control-display gain corresponding to the first angle range.
9. The method of claim 1 , wherein the property is a velocity of the hand.
10. The method of claim 9 , wherein:
identifying the velocity of the hand comprises:
determining a location of the hand;
storing the location of the hand in a queue that includes one or more locations of the hand while the gesture-controlled device is in a drag state; and
determining the velocity of the hand based on the one or more locations of the hand stored in the queue;
adjusting the control-display comprises:
in response to determining that the velocity has been below a velocity threshold for at least a dwell time threshold, and that the gesture-controlled device is in a first control-display gain state of a plurality of control-display gain states:
placing the gesture-controlled device into a second control-display gain state of the plurality of control-display gain states; and
adjusting the control-display gain to a control-display gain corresponding to the second control-display gain state.
11. The method of claim 1 , wherein:
identifying the velocity of the hand comprises:
determining a location of the hand;
storing the location of the hand in a queue that includes one or more locations of the hand while the gesture-controlled device is in a drag state; and
determining the velocity of the hand based the one or more locations of the hand stored in the queue; and
the control-display gain is determined in accordance with a function dependent on the velocity of the hand.
12. The method of claim 11 , wherein the function dependent on the velocity is a linear function.
13. The method of claim 11 , wherein the function dependent on the velocity is a generalized logistic function.
14. A gesture-controlled device comprising:
a processor device; and
a memory storing machine-executable instructions thereon which, when executed by the processing device, cause the gesture-controlled device to:
processing a video frame to detect a location and shape of a hand in the video frame;
generating gesture data for the video frame based on the location and shape of the hand;
determining that the hand is performing a dragging hand gesture based on gesture data for the video frame and gesture data for one or more previous video frames;
identify a property of a hand of a user of the gesture-controlled device performing the dragging hand gesture; and
adjust a control-display gain for the gesture-controlled device to a control-display gain determined based on the identified property.
15. The gesture-controlled device of claim 14 , wherein the property is the location of the hand.
16. The gesture-controlled device of claim 14 , wherein the property is an orientation of the hand relative to a reference orientation.
17. The gesture-controlled device of claim 14 , wherein the property is a velocity of the hand.
18. The gesture-controlled device of claim 17 , wherein:
identifying the velocity of the hand comprises:
determining a location of the hand;
storing the location of the hand in a queue that includes one or more locations of the hand while the gesture-controlled device is in a drag state; and
determining the velocity of the hand based the one or more locations of the hand stored in the queue;
adjusting the control-display comprises:
in response to determining that the velocity has been below a velocity threshold for at least a dwell time threshold, and that the gesture-controlled device is in a first control-display gain state of a plurality of control-display gain states:
placing the gesture-controlled device into a second control-display gain state of the plurality of control-display gain states; and
adjusting the control-display gain to a control-display gain corresponding to the second control-display gain state.
19. The gesture-controlled device of claim 17 , wherein:
identifying the velocity of the hand comprises:
determining a location of the hand;
storing the location of the hand in a queue that includes one or more locations of the hand while the gesture-controlled device is in a drag state; and
determining the velocity of the hand based the one or more locations of the hand stored in the queue; and
the control-display gain is determined in accordance with a function dependent on the velocity of the hand.
20. A non-transitory processor-readable medium having machine-executable instructions stored thereon which, when executed by a processor of a gesture-controlled device, cause the processor device to:
process a video frame to detect a location and shape of a hand mid-air hand gesture in the video frame;
generate gesture data for the video frame based on the location and shape of the hand;
determine that the mid-air hand is performing hand gesture is a dragging hand gesture based on gesture data for the video frame and gesture data for one or more previous video frames;
identify a property of a hand of a user of the gesture-controlled device performing the dragging hand gesture; and
adjust a control-display gain for the gesture-controlled device to a control-display gain determined based on the identified property.