Granted Patent B2

US 10,423,226 · App. 14/616,603 · Granted Sep 24, 2019

Abstract

Claims (69)

1. A method of providing visual feedback for an interaction between a control object in a three-dimensional (3D) sensory space and a virtual object in a virtual space that the control object interacts with, the method including:

defining a virtual feeler zone for at least a portion of a human hand that serves as a control object that is tracked by a gesture-recognition system while moving in the three-dimensional (3D) sensory space;

generating for display, a feeler proximity-pressure indicator that provides visual feedback including:

a visual indication of proximity of the virtual feeler zone to the virtual object; and

a visual indication of escalation from proximity to pressure applied by virtual contact of the virtual feeler zone with the virtual object;

modifying presentation of the feeler proximity-pressure indicator responsive to changes in distance between the virtual object and a tracked position of the at least a portion of a human hand that serves as a control object detected by the gesture-recognition system while the at least a portion of a human hand that serves as a control object moves in the three-dimensional (3D) sensory space; and

presenting to a user the feeler proximity-pressure indicator modified; thereby indicating that the virtual object is manipulated responsive to virtual contact with the at least a portion of a human hand that serves as a control object,

wherein the gesture-recognition system uses only stereoscopic vision to detect motion of the hand as it moves freely in space without contacting any other physical objects.

2. The method of claim 1 , wherein the virtual feeler zone is a virtual extension of the control object.

3. The method of claim 1 , wherein the virtual feeler zone is a deformable region of the control object.

4. The method of claim 1 , wherein the feeler proximity-pressure indicator is located at a distal end of the virtual feeler zone.

5. The method of claim 1 , further including snapping the feeler proximity-pressure indicator to a surface of the virtual object when the virtual feeler zone approaches the virtual object within an initial hover proximity threshold.

6. The method of claim 1 , further including modifying appearance of the feeler proximity-pressure indicator responsive to distance between the virtual feeler zone and the virtual object.

7. The method of claim 1 , further including modifying size of the feeler proximity-pressure indicator responsive to distance between the virtual feeler zone and the virtual object.

8. The method of claim 1 , further including modifying shape of the feeler proximity-pressure indicator responsive to distance between the virtual feeler zone and the virtual object.

9. The method of claim 1 , further including modifying opacity of the feeler proximity-pressure indicator responsive to distance between the virtual feeler zone and the virtual object.

10. The method of claim 1 , further including applying a spring function to calculate pressure applied by virtual contact over a compressible range.

11. The method of claim 1 , further including:

tracking a speed of approach of the virtual feeler zone of the control object to a surface of the virtual object; and

adjusting an initial hover proximity threshold and calibration of the visual feedback responsive to the speed of approach.

12. The method of claim 11 , wherein the initial hover proximity threshold distance is adjusted proportionally to the speed of approach.

13. The method of claim 1 , wherein the virtual object is movable when the virtual feeler zone applies pressure and not deformable, further including:

responsive to virtual contact of the virtual feeler zone with the virtual object, depicting for display, displacement of the virtual object without depicting penetration of the virtual object by the virtual feeler zone, while providing the visual feedback regarding the pressure applied by virtual contact; and

providing calculated pressure applied by virtual contact as input to an object movement system.

14. The method of claim 1 , wherein the virtual object is deformable when the virtual feeler zone applies pressure, further including:

responsive to a period of virtual contact of the virtual feeler zone with the virtual object, depicting for display, deformation of the virtual object without depicting penetration of the virtual object by the virtual feeler zone, while providing the visual feedback regarding the pressure applied by virtual contact; and

providing calculated pressure applied by virtual contact as input to an object movement system.

15. The method of claim 1 , wherein the virtual object is movable and deformable when the virtual feeler zone applies force, further including

responsive to a period of virtual contact of the virtual feeler zone with the virtual object, depicting for display, displacement and deformation of the virtual object without depicting penetration of the virtual object by the virtual feeler zone, while providing the visual feedback regarding pressure applied by virtual contact; and

providing calculated pressure applied by virtual contact as input to an object movement system.

16. The method of claim 1 , wherein the virtual object is immovable and not deformable when the virtual feeler zone applies force, further including

responsive to a continuous virtual contact of the virtual feeler zone with the virtual object, depicting for display, inertia and non-deformability of the virtual object without depicting penetration of the virtual object by the virtual feeler zone, while providing the visual feedback regarding pressure applied by virtual contact; and

providing calculated pressure applied by virtual contact as input to an object movement system.

17. The method of claim 1 , wherein the virtual object has a surface texture, further including generating for display a feeler surface-roughness indicator that provides visual feedback regarding roughness of the surface texture.

18. The method of claim 1 , wherein the virtual object is a rigid solid, further including generating for display a feeler object-mass indicator that provides visual feedback regarding mass of the rigid solid.

19. The method of claim 1 , wherein the virtual object is a volumetric solid, further including generating for display a feeler object-density indicator that provides visual feedback regarding density of the volumetric solid.

20. The method of claim 1 , wherein the virtual object is a fluid that conforms to the control object, further including generating for display a feeler object-viscosity indicator that provides visual feedback regarding viscosity of the fluid.

21. The method of claim 1 , wherein the virtual object has a surface temperature, further including generating for display a feeler surface-temperature indicator that provides visual feedback regarding hotness or coldness of a surface of the virtual object.

22. The method of claim 1 , further including:

calculating opposing forces applied by a plurality of feeler zones to a movable virtual object; and

generating for display a feeler grip-indicator that provides visual feedback regarding whether opposing forces applied by a plurality of feeler zones are sufficient to grasp the virtual object proportional to mass of the virtual object.

23. The method of claim 1 , further including:

calculating opposing forces applied by a plurality of feeler zones to a movable virtual object; and

generating for display a feeler grip-indicator that provides visual feedback regarding whether opposing forces applied by a plurality of feeler zones are sufficient to grasp the virtual object proportional to density of the virtual object.

24. The method of claim 1 , further including:

calculating opposing forces applied by a plurality of feeler zones to a movable virtual object; and

generating for display a feeler grip-indicator that provides visual feedback regarding whether opposing forces applied by a plurality of feeler zones are sufficient to grasp the virtual object proportional to surface texture of the virtual object.

25. The method of claim 1 , further including the human hand grasping the virtual object; and

modifying presentation of the feeler proximity-pressure indicator responsive to grasping to provide a visual indication of escalation from proximity to pressure applied by virtual contact of the virtual feeler zone with the virtual object during grasping.

26. The method of claim 1 , further including the human hand being empty.

27. A system for providing visual feedback for an interaction between a control object in a three-dimensional (3D) sensory space and a virtual object in a virtual space that the control object interacts with, the system including:

a processor and a computer readable storage medium storing computer instructions configured to cause the processor to:

define a virtual feeler zone for at least a portion of a human hand that serves as a control object that is tracked by a gesture-recognition system while moving in the three-dimensional (3D) sensory space;

generate for display, a feeler proximity-pressure indicator that provides visual feedback including:

a visual indication of proximity of the virtual feeler zone to the virtual object; and

a visual indication of escalation from proximity to pressure applied by virtual contact of the virtual feeler zone with the virtual object;

modify presentation of the feeler proximity-pressure indicator responsive to changes in distance between the virtual object and a tracked position of the at least a portion of a human hand that serves as a control object detected by the gesture-recognition system while the at least a portion of a human hand that serves as a control object moves in the three-dimensional (3D) sensory space; and

present to a user the feeler proximity-pressure indicator modified; thereby indicating that the virtual object is manipulated responsive to virtual contact with the at least a portion of a human hand that serves as a control object,

wherein the gesture-recognition system uses only stereoscopic vision to detect motion of the hand as it moves freely in space without contacting any other physical objects.

28. The system of claim 27 , further including the human hand shaking the virtual object; and

modifying presentation of the feeler proximity-pressure indicator responsive to shaking to provide a visual indication of escalation from proximity to pressure applied by virtual contact of the virtual feeler zone with the virtual object during shaking.

29. A non-transitory computer readable medium impressed with instructions to provide visual feedback for an interaction between a control object in a three-dimensional (3D) sensory space and a virtual object in a virtual space that the control object interacts with, which instructions when executed by one or more processors implement a method including:

defining a virtual feeler zone for at least a portion of a human hand that serves as a control object that is tracked by a gesture-recognition system while moving in the three-dimensional (3D) sensory space;

generating for display, a feeler proximity-pressure indicator that provides visual feedback including:

a visual indication of proximity of the virtual feeler zone to the virtual object; and

a visual indication of escalation from proximity to pressure applied by virtual contact of the virtual feeler zone with the virtual object;

modifying presentation of the feeler proximity-pressure indicator responsive to changes in distance between the virtual object and a tracked position of the at least a portion of a human hand that serves as a control object detected by the gesture-recognition system while the at least a portion of a human hand that serves as a control object moves in the three-dimensional (3D) sensory space; and

presenting to a user the feeler proximity-pressure indicator modified; thereby indicating that the virtual object is manipulated responsive to virtual contact with the at least a portion of a human hand that serves as a control object,

wherein the gesture-recognition system uses only stereoscopic vision to detect motion of the hand as it moves freely in space without contacting any other physical objects.

Assignments (15)