Velocity field interaction for free space gesture interface and control
View Patent ↗The technology disclosed relates to automatically interpreting a gesture of a control object in a three dimensional sensor space by sensing a movement of the control object in the three dimensional sensor space, sensing orientation of the control object, defining a control plane tangential to a surface of the control object and interpreting the gesture based on whether the movement of the control object is more normal to the control plane or more parallel to the control plane.
1. A method of automatically interpreting a gesture of a control object, in a three-dimensional (3D) sensor space using a 3D sensor, as a first gesture or a second gesture, the method including:
defining a control plane that remains tangent to a surface of the control object throughout a sensed movement of the control object in any direction in the 3D sensor space, the control plane being defined by an orientation of the control object, as sensed by a video camera of the 3D sensor;
interpreting the gesture of the control object as the first gesture or the second gesture by comparing a direction of a sensed trajectory of the movement of the control object to (i) a normal vector that is normal to the defined control plane and (ii) a surface of the defined control plane, wherein:
the gesture is the first gesture when the direction of the trajectory of the movement of the control object is within a pre-determined range of the normal vector of the control plane; and
the gesture is the second gesture when the direction of the trajectory of the movement of the control object is parallel to the surface of the control plane, within a pre-determined range,
such that the gesture is the first gesture when the direction of the movement of the trajectory of the control object is more normal to the surface of the control plane than parallel to the surface of the control plane and the gesture is the second gesture when the direction of the movement of the trajectory of the control object is more parallel to the surface of the control plane than normal to the surface of the control plane.
2. The method of claim 1 , wherein:
the control object is a hand; and
the control plane is tangent to a palm or back of the hand.
3. The method of claim 1 , wherein the interpreting of the gesture includes:
calculating an angular trajectory of the movement of the control object; and
determining whether the gesture engages a virtual control based on whether the trajectory is normal or parallel to the control plane.
4. The method of claim 1 , further including determining the control plane is normal to the trajectory when the normal vector of the control plane is within +/−10 degrees from a tangent vector of the trajectory.
5. The method of claim 1 , wherein the first gesture is an interaction gesture for interacting with an object and the second gesture in a non-interaction gesture.
6. A method of automatically interpreting a gesture of a control object, in a three-dimensional (3D) sensor space relative to a flow depicted in a display, as a first gesture or a second gesture, the method including:
defining a control plane that remains tangent to a surface of the control object throughout a sensed movement of the control object in any direction in the 3D sensor space, the control plan being defined by an orientation of the control object, as sensed by a video camera of the 3D sensor; and
interpreting the gesture of the control object as the first gesture or the second gesture by comparing a direction of the flow to (i) a normal vector that is normal to the defined control plane and (ii) a surface of the defined control plane, wherein:
the gesture is the first gesture when the direction of the flow is within a pre-determined range of the normal vector of the control plane; and
the gesture is the second gesture when the direction of the flow is parallel to the surface of the control plane, within a pre-determined range,
such that the gesture is the first gesture when the direction of the flow is more normal to the surface of the control plane than parallel to the surface of the control plane, and the gesture is the second gesture when the direction of the flow is more parallel to the surface of the control plane than normal to the surface of the control plane.
7. The method of claim 6 , wherein:
the control object is a hand; and
the control plane is tangent to a palm or back of the hand.
8. The method of claim 6 , wherein the interpreting of the gesture includes:
calculating a velocity and weighted velocity of the movement of the control object, the weighted velocity being weighted by a projection of the velocity onto a direction parallel to the flow depicted in the display and by a projection of a normal vector of the control plane onto the direction parallel to the flow depicted in the display; and
determining an amount the gesture engages a virtual control of the flow depicted in the display based on the calculated velocity.
9. The method of claim 6 , further including determining an amount that a hand gesture engages vectors describing the flow using the normal vector based on whether the control plane and movement of the hand are determined to be normal or parallel to the vectors of the flow.
10. A method of navigating a multi-layer presentation tree using gestures of a control object in a three-dimensional (3D) sensor space, using a 3D sensor, by distinguishing between the control object and a sub-object of the control object, the method including:
processing an output of a video camera of the 3D sensor thereby sensing a movement of the control object in any direction in the 3D sensor space using a control plane that remains tangent to a surface of the control object throughout the movement of the control object in any direction in the 3D sensor space, the control plane being defined by processing the output of the video camera of the 3D sensor and a sensed orientation of the control object; and
interpreting by a computing device a direction of the movement of the control object as scrolling through a particular level of the multi-layer presentation tree when the direction of the movement of the control object is more normal with respect to the surface of the tangent control plane than parallel with respect to the surface of the control plane.
11. The method of claim 10 , wherein the sub-object is a portion of the control object.
12. The method of claim 10 , wherein the control object is a hand and the sub-object is a finger of the hand.
13. The method of claim 10 , further including: (i) traversing menus responsive to movement of a first sub-object including a first hand and (ii) traversing menu paths responsive to movement of a second sub-object including a second hand.
14. A method of interpreting a gesture of a control object in a 3D sensor space relative to one or more objects depicted in a display, the method including:
sensing a speed of a movement of the control object moving in any direction through the 3D sensor space using a 3D sensor;
interpreting by a computing device the movement as a path on the display if the speed of the movement exceeds a pre-determined threshold; and
duplicating one or more of the objects in the display that intersect the path as interpreted.
15. The method of claim 14 , wherein the objects are pre-selected.
16. The method of claim 14 , further including the movement as a path on the display if the speed of the movement exceeds a pre-determined threshold of 20 cm per second.