Interaction strength using virtual objects for machine control
The technology disclosed relates to using virtual attraction between hand or other control object in a three-dimensional (3D) sensory space and a virtual object in a virtual space. In particular, it relates to defining a virtual attraction zone of a hand or other control object that is tracked in a three-dimensional (3D) sensory space and generating one or more interaction forces between the control object and a virtual object in a virtual space that cause motion of the virtual object responsive to proximity of the control object to the virtual object and escalation with a virtual pinch or grasp action of the control object directed to a manipulation point of the virtual object.
1. A method of using virtual attraction between a hand in a three-dimensional (3D) sensory space and a virtual object in a virtual space, the method including:
defining a virtual attraction zone of a hand that is tracked in a three-dimensional (3D) sensory space;
creating a manipulation point of the hand responsive to a virtual pinch action between two fingers of the hand, wherein a movement of the created manipulation point follows a movement of the hand, and wherein the created manipulation point remains within a proximity of the hand as the hand moves throughout the 3D sensory space;
selecting a virtual object as the previously created manipulation point of the hand comes within a proximity of the virtual object or a manipulation point of the virtual object, wherein at least the two fingers of the hand are not within a zero proximity distance from the virtual object; and
generating one or more interaction forces between the hand and the selected virtual object in a virtual space, which cause motion of the selected virtual object, in response to:
movement of the manipulation point of the hand with respect to the selected virtual object; and
escalation of the virtual pinch action of the hand with respect to the selected virtual object.
2. The method of claim 1 , further including:
generating a predictive model of the hand, and using the predictive model to track motion of the hand, wherein the predictive model includes positions of calculation points of fingers, thumb and palm of the hand;
dynamically selecting at least one manipulation point proximate to the virtual object based on the motion tracked by the predictive model and positions of one or more of the calculation points; and
manipulating the virtual object by interaction between at least some of the calculation points of the predictive model and the dynamically selected manipulation point.
3. The method of claim 2 , wherein the predictive model is generated based on at least one of a feature of interest of the hand and a salient feature or property of an image used to capture the feature of interest of the hand.
4. The method of claim 3 , wherein the feature of interest is based on a brightness of a portion of the hand captured in the image.
5. The method of claim 2 , further including applying a constraint factor to the predictive model to eliminate impossible poses of the hand based on one or more physical properties of the hand.
6. The method of claim 2 , further including:
determining an outside pinch pose responsive to de-escalation of distance between opposable calculation points of the thumb and at least one of the fingers during dynamic selection of the manipulation point;
assigning a virtual attraction strength attribute to the outside pinch pose based on a degree of convergence of the calculation points; and
manipulating the virtual object responsive to the virtual attraction strength attribute assigned to the outside pinch pose when the calculation points are within a predetermined range of each other.
7. The method of claim 2 , further including:
determining an inside pinch pose responsive to escalation of distance between opposable calculation points of the thumb and at least one of the fingers during dynamic selection of the manipulation point;
assigning a virtual attraction strength attribute to the inside pinch pose based on a degree of convergence of the calculation points; and
manipulating the virtual object responsive to the virtual attraction strength attribute assigned to the inside pinch pose when the calculation points are within a predetermined range of each other.
8. The method of claim 2 , further including:
determining a grab pose for the predictive model based on convergence of one or more of the calculation points of the fingers and the palm of the hand during dynamic selection of the manipulation point;
assigning a virtual attraction strength attribute to the grab pose based on a degree of convergence of the calculation points; and
manipulating the virtual object responsive to the virtual attraction strength attribute assigned to the grab pose when the calculation points are within a predetermined range of each other.
9. The method of claim 8 , further including manipulating the virtual object responsive to the virtual attraction strength attribute assigned to the grab pose exceeding at least one of a floor value and a band of permissible levels.
10. The method of claim 2 , further including generating data for augmented display representing a position of the virtual object relative to the predictive model of the hand.
11. The method of claim 2 , further including generating data for display representing positions in a rendered virtual space of the virtual object and the predictive model of the hand.
12. The method of claim 11 , further including manipulating the virtual object responsive to a proximity between at least some of the calculation points of the predictive model and the manipulation point of the virtual object.
13. The method of claim 2 , further including:
determining a plurality of poses for the predictive model based on simultaneous convergence of one or more of the calculation points of one or more hands during dynamic selection of the manipulation point;
assigning virtual attraction strength attributes to respective poses in the plurality based on a degree of convergence of corresponding calculation points;
identifying a dominant pose from the plurality based on at least one of magnitudes of the strength attributes and positions of the simultaneous convergence; and
manipulating the virtual object responsive to the dominant pose.
14. The method of claim 2 , further including:
determining a plurality of poses for the predictive model based on simultaneous convergence of one or more of the calculation points of one or more hands during dynamic selection of the manipulation point;
assigning virtual attraction strength attributes to respective poses in the plurality based on a degree of convergence of corresponding calculation points;
calculating a combined virtual attraction based on adding the virtual attraction strength attributes of the respective poses; and
applying interaction forces to the virtual object based on the combined virtual attraction.
15. The method of claim 1 , wherein acceleration of the virtual object is proportionally responsive to escalation with the virtual pinch action of the hand.
16. The method of claim 1 , wherein deceleration of the virtual object is proportionally responsive to escalation with the virtual pinch action of the hand.
17. A method of using virtual force applied by a hand in a three-dimensional (3D) sensory space to a virtual object with a virtual mass in a virtual space, the method including:
defining a virtual force applied by a hand that is tracked in a three-dimensional (3D) sensory space;
creating a manipulation point of the hand responsive to, a virtual grasp action between fingers of the hand and a palm of a hand or a swiping action of the hand, wherein a movement of the created manipulation point follows a movement of the hand, and wherein the created manipulation point remains within a proximity of the hand as the hand moves throughout the 3D sensory space;
selecting a virtual object as the created manipulation point of the hand comes within a proximity of the virtual object or a manipulation point of the virtual object, wherein at least the fingers of the hand are not within a zero proximity distance from the virtual object; and
generating one or more interaction forces between the hand and the selected virtual object with a virtual mass in a virtual space, wherein the interaction forces cause motion of the selected virtual object in response to:
movement of the manipulation point of the hand with respect to the selected virtual object; and
escalation of the virtual grasp action or the swipe action of the hand with respect to the selected virtual object.
18. The method of claim 17 , further including:
generating a predictive model of the hand, and using the predictive model to track motion of the hand, wherein the predictive model includes positions of calculation points of fingers, thumb and palm of the hand;
dynamically selecting at least one manipulation point proximate to the virtual object based on the motion tracked by the predictive model and positions of one or more of the calculation points; and
manipulating the virtual object by interaction between at least some of the calculation points of the predictive model and the dynamically selected manipulation point.
19. The method of claim 18 , wherein the predictive model is generated based on least one of a feature of interest of the hand and a salient feature or property of an image used to capture the feature of interest of the hand.
20. The method of claim 19 , wherein the feature of interest is based on a brightness of a portion of the hand captured in the image.
21. The method of claim 18 , further including applying a constraint factor to the predictive model to eliminate impossible poses of the hand based on one or more physical properties of the hand.
22. The method of claim 18 , further including:
determining a grab pose for the predictive model based on convergence of one or more of the calculation points of the fingers and the palm of the hand during dynamic selection of the manipulation point;
assigning a virtual force strength attribute to the grab pose based on a degree of convergence of the calculation points; and
manipulating the virtual object responsive to the virtual force strength attribute assigned to the grab pose when the calculation points are within a predetermined range of each other.
23. A method of using virtual repulsion between a hand in a three-dimensional (3D) sensory space and a virtual object in a virtual space, the method including:
defining a virtual repulsion zone of a hand that is tracked in a three-dimensional (3D) sensory space;
creating a manipulation point of the hand responsive to a virtual pinch action between two fingers of the hand, wherein a movement of the created manipulation point follows a movement of the hand, and wherein the created manipulation point remains within a proximity of the hand as the hand moves throughout the 3D sensory space;
selecting a virtual object as the previously created manipulation point of the hand comes within a proximity of the virtual object or a manipulation point of the virtual object, wherein at least the two fingers of the hand are not within a zero proximity distance from the virtual object; and
generating one or more repulsion interaction forces between the hand and the selected virtual object in a virtual space, which cause motion of the selected virtual object, in response to:
movement of the manipulation point of the hand with respect to the selected virtual object; and
escalation of the virtual pinch action of the hand with respect to the selected virtual object.
24. The method of claim 23 , wherein acceleration of the virtual object is proportionally responsive to escalation with the virtual pinch action of the hand.
25. The method of claim 23 , wherein deceleration of the virtual object is proportionally responsive to escalation with the virtual pinch action of the hand.
26. The method of claim 19 , wherein the virtual object is a scalpel and manipulating the virtual object further includes slicing using the virtual object.
27. The method of claim 1 , further comprising creating an anchor point at a location on the hand responsive to the virtual pinch action between the two fingers of the hand, wherein the created manipulation point remains within a proximity of the anchor point at the location on the hand as the hand moves throughout the 3D sensory space.