Method and device for adjusting the control-display gain of a gesture controlled electronic device
Methods, devices, and processor-readable media for adjusting the control-display gain of a gesture-controlled device are described. Adjusting the control-display gain may facilitate user interaction with content or UI elements rendered on a display screen of the gesture-controlled device. The control-display gain may be adjusted based on a property of how a mid-air dragging gesture is being performed by a user's hand. The property may be the location of the gesture, the orientation of the hand performing the gesture, or the velocity of the gesture. A hand that becomes stationary for a threshold time period while performing the dragging gesture may adjust the control-display gain to a different level. Control-display gain may be set to a different value based on the current velocity of the hand performing the gesture. The control-display gain levels may be selected from a continuous range of values or a set of discrete values. Devices for performing the methods are described.
1. A method for controlling a gesture-controlled device, the method comprising:
obtaining a plurality of video frames;
for each video frame of the plurality of video frames:
processing the video frame to detect a shape of a hand in the video frame; and
generating gesture data for the video frame based on the shape of the hand;
processing the gesture data for each video frame of the plurality of video frames to determine that the hand is performing a hand gesture;
adjusting a control-display gain function for the gesture-controlled device based on at least one of a location of the hand performing the hand gesture, and an orientation of the hand performing the hand gesture relative to a reference orientation.
2. The method of claim 1 , the method further comprising:
storing the location of the hand in a queue that includes one or more locations of the hand; and
determining a movement of the hand with respect to a first axis based on the one or more locations of the hand stored in the queue;
wherein the control-display gain function is determined based on the location of the hand with respect to a second axis orthogonal to the first axis.
3. The method of claim 1 , wherein:
identifying the location of the hand comprises determining that the hand is located within a first region of the video frame; and
the control-display gain function is a first control-display gain function corresponding to the first region.
4. The method of claim 1 , wherein the orientation of the hand comprises an angle of the hand relative to the reference orientation.
5. The method of claim 4 , wherein the angle of the hand is measured with respect to a vertical plane.
6. The method of claim 4 , wherein:
identifying the orientation of the hand comprises determining that the angle of the hand is within a first angle range defined between a first angle and a second angle; and
adjusting the control-display gain function comprises adjusting the control-display gain function to a first control-display gain function corresponding to the first angle range.
7. The method of claim 1 , further comprising adjusting the control-display gain function for the gesture-controlled device based on a velocity of the hand.
8. The method of claim 7 , wherein:
identifying the velocity of the hand comprises:
determining a location of the hand;
storing the location of the hand in a queue that includes one or more locations of the hand; and
determining the velocity of the hand based on the one or more locations of the hand stored in the queue.
9. The method of claim 8 , wherein:
adjusting the control-display gain function comprises:
in response to determining that the velocity has been below a velocity threshold for at least a dwell time threshold, and that the gesture-controlled device is in a first control-display gain state of a plurality of control-display gain states:
placing the gesture-controlled device into a second control-display gain state of the plurality of control-display gain states; and
adjusting the control-display gain function to a control-display gain function corresponding to the second control-display gain state.
10. The method of claim 1 , further comprising:
obtaining a further plurality of video frames;
processing the plurality of video frames to detect a velocity of a hand within the plurality of video frames; and
applying the control-display gain function to the detected velocity of the hand to determine a cursor velocity; and
displaying the cursor on a display screen of the gesture-controlled device, the cursor being displayed to exhibit the cursor velocity.
11. The method of claim 10 , wherein the control-display gain function is a constant control-gain level, such that the cursor velocity is at a constant proportion to the velocity of the hand.
12. The method of claim 10 , wherein the control-display gain function is a linear function of the velocity of the hand, such that the cursor velocity is at a proportion to the velocity of the hand linearly varying in proportion to the velocity of the hand.
13. The method of claim 10 , wherein the control-display gain function is a generalized logistic function of the velocity of the hand, such that the cursor velocity is at a proportion to the velocity of the hand varying as a generalized logistic function of the velocity of the hand.
14. A gesture-controlled device comprising:
a processor device; and
a memory storing machine-executable instructions thereon which, when executed by the processing device, cause the gesture-controlled device to:
obtain a plurality of video frames;
for each video frame of the plurality of video frames:
process the video frame to detect a shape of a hand in the video frame; and
generate gesture data for the video frame based on the shape of the hand;
process the gesture data for each video frame of the plurality of video frames to determine that the hand is performing a hand gesture;
adjust a control-display gain function for the gesture-controlled device based on at least one of a location of the hand performing the hand gesture, and an orientation of the hand performing the hand gesture relative to a reference orientation.
15. The gesture-controlled device of claim 14 , further comprising adjusting the control-display gain function based on a velocity of the hand.
16. A non-transitory processor-readable medium having machine-executable instructions stored thereon which, when executed by a processor of a gesture-controlled device, cause the processor device to:
obtain a plurality of video frames;
for each video frame of the plurality of video frames:
process the video frame to detect a shape of a hand in the video frame; and
generate gesture data for the video frame based on the shape of the hand;
process the gesture data for each video frame of the plurality of video frames to determine that the hand is performing a hand gesture;
adjust a control-display gain function for the gesture-controlled device based on at least one of a location of the hand performing the hand gesture, and an orientation of the hand performing the hand gesture relative to a reference orientation.