Interfacing with a device via virtual 3D objects
The present invention provides a system and method for interacting with a 3D virtual image containing activatable objects. The system of the invention includes a 3D display device that presents to a user a 3D image, an image sensor and a processor. The processor analyzes images obtained by the image sensor to determine when the user has placed an activating object such as a hand or a finger, or has performed a gesture related to an activatable object, in the 3D space, at the location where the user perceives an activatable object to be located. The user thus perceives that he is “touching” the activatable object with the activating object.
1. A touch-free gesture recognition system, comprising:
at least one processor configured to:
display a virtual 3D image that includes at least one activatable object, the activatable object having at least two distinct sides that are simultaneously activatable, wherein a first distinct side of the at least two distinct sides is associated with a first action, and a second distinct side of the at least two distinct sides is associated with a second action;
receive, from at least one image sensor, at least one image including a depiction of an activating object associated with a user;
correlate a location of the activating object with a location of one of the first and second distinct sides; and
implement the first action when the location of the activating object correlates to the first distinct side and implement the second action when the location of the activating object correlates to the second distinct side.
2. The touch-free gesture recognition system of claim 1 , wherein the first action includes at least one of a first message or a first command, and the second action includes at least one of a second message or a second command.
3. The touch-free gesture recognition system of claim 1 , wherein the first and second distinct sides are each associated with at least one of a left side of the activatable object, a right side of the activatable object, a bottom side of the activatable object, and a top side of the activatable object.
4. The touch-free gesture recognition system of claim 3 , wherein the at least one processor is further configured to:
detect a predetermined motion associated with the activating object including at least one of a right side click, a left side click, a bottom side click, a top side click, a grasp of the activatable object, a gesture toward the activatable object from the left side, a gesture toward the activatable object from the right side, and a pass through the activatable object.
5. The touch-free gesture recognition system of claim 1 , wherein the at least one processor is further configured to:
detect a voice command and select the first action and the second action, based, at least in part, on the voice command.
6. The touch-free gesture recognition system of claim 1 , wherein the first and second distinct sides are each associated with at least one of a light touch to the activatable object and a deep touch to the activatable object.
7. The touch-free gesture recognition system of claim 1 , wherein the at least one processor is further configured to:
detect the activatable object based, at least in part, on eye movements of the user.
8. The touch-free gesture recognition system of claim 1 , wherein the at least one processor is further configured to:
adjust the display of the virtual 3D image as the activating object moves closer to the activatable object.
9. The touch-free gesture recognition system of claim 1 , wherein, to correlate the location of the activating object with one of the first and second distinct sides, the at least one processor is further configured to:
determine whether the activating object is in a virtual cone that has a vertex associated with the image sensor and that extends through at least a portion of the activatable object.
10. The touch-free gesture recognition system of claim 9 , wherein, to correlate the location of the activating object with one of the first and second distinct sides, the at least one processor is further configured to:
determine whether the activating object is less than a predetermined distance from at least one of the activatable object or a display device associated with the activatable object.
11. The touch-free gesture recognition system of claim 1 , wherein, to correlate the location of the activating object with one of the first and second distinct sides, the at least one processor is further configured to:
determine whether the activating object is pointing at the activatable object and is less than a predetermined distance from the activatable object.
12. The touch-free gesture recognition system of claim 1 , wherein, to correlate the location of the activating object with one of the first and second distinct sides, the at least one processor is further configured to:
determine whether the activating object is pointing at the activatable object and is less than a predetermined distance from a display device associated with the activatable object.
13. The touch-free gesture recognition system of claim 1 , wherein, to correlate the location of the activating object with one of the first and second distinct sides, the at least one processor is further configured to:
determine whether the activating object is performing a predetermined gesture and is less than a predetermined distance from the activatable object.
14. The touch-free gesture recognition system of claim 1 , wherein, to correlate the location of the activating object with one of the first and second distinct sides, the at least one processor is further configured to:
determine whether the activating object is performing a predetermined gesture and is less than a predetermined distance from a display device associated with the activatable object.
15. A non-transitory computer-readable medium including instructions that, when executed by at least one processor, cause the processor to perform operations, comprising:
displaying a virtual 3D image that includes at least one activatable object, the activatable object having at least two distinct sides that are simultaneously activatable, wherein a first distinct side of the at least two distinct sides is associated with a first action and a second distinct side of the at least two distinct sides is associated with a second action;
receiving, from at least one image sensor, at least one image including a depiction of an activating object associated with a user;
correlating a location of the activating object with a location of one of the first and second distinct sides;
if the location of the activating object correlates to the first distinct side, implementing the first action, and
if the location of the activating object correlates to the second distinct side, implementing the second action.
16. The non-transitory computer-readable medium of claim 15 , wherein the first action includes at least one of a first message or a first command, and the second action includes at least one of a second message or a second command.
17. The non-transitory computer-readable medium of claim 15 , wherein the first and second distinct sides are each associated with at least one of a left side of the activatable object, a right side of the activatable object, a bottom side of the activatable object, and a top side of the activatable object.
18. The non-transitory computer-readable medium of claim 17 , wherein the instructions, when executed by at least one processor, further cause the processor to perform at least one operation comprising:
detecting a predetermined motion associated with the activating object including at least one of a right side click, a left side click, a bottom side click, a top side click, a grasp of the activatable object, a gesture toward the activatable object from the left side, a gesture toward the activatable object from the right side, and a pass through the activatable object.
19. The non-transitory computer-readable medium of claim 15 , wherein the instructions, when executed by at least one processor, further cause the processor to perform at least one operation comprising:
detecting a voice command; and
selecting at least one of the first action and the second action, based, at least in part, on the voice command.
20. The non-transitory computer-readable medium of claim 15 , wherein the first and second distinct sides are each associated with at least one of a light virtual touch to the activatable object and a deep virtual touch to the activatable object.
21. The non-transitory computer-readable medium of claim 15 , wherein the instructions, when executed by at least one processor, further cause the processor to perform at least one operation comprising:
detecting the activatable object based, at least in part, on an eye movement of the user.
22. The non-transitory computer-readable medium of claim 15 , wherein the instructions, when executed by at least one processor, further cause the processor to perform at least one operation comprising:
adjusting the display of the virtual 3D image as the activating object moves closer to the activatable object.
23. The non-transitory computer-readable medium of claim 15 , wherein, to correlate the location of the activating object with one of the first and second distinct sides, the instructions, when executed by at least one processor, further cause the processor to perform at least one operation comprising:
determining whether the activating object is in a virtual cone that has a vertex associated with the image sensor and that extends through at least a portion of the activatable object.
24. The non-transitory computer-readable medium of claim 23 , wherein, to correlate the location of the activating object with one of the first and second distinct sides, the instructions, when executed by at least one processor, further cause the processor to perform at least one operation comprising:
determining whether the activating object is less than a predetermined distance from at least one of the activatable object or a display device associated with the activatable object.
25. The non-transitory computer-readable medium of claim 15 , wherein, to correlate the location of the activating object with one of the first and second distinct sides, the instructions, when executed by at least one processor, further cause the processor to perform at least one operation comprising:
determining whether the activating object is pointing at the activatable object and is less than a predetermined distance from the activatable object.
26. The non-transitory computer-readable medium of claim 15 , wherein, to correlate the location of the activating object with one of the first and second distinct sides, the instructions, when executed by at least one processor, further cause the processor to perform at least one operation comprising:
determining whether the activating object is pointing at the activatable object and is less than a predetermined distance from a display device associated with the activatable object.
27. The non-transitory computer-readable medium of claim 15 , wherein, to correlate the location of the activating object with one of the first and second distinct sides, the instructions, when executed by at least one processor, further cause the processor to perform at least one operation comprising:
determining whether the activating object is performing a predetermined gesture and is less than a predetermined distance from the activatable object.
28. The non-transitory computer-readable medium of claim 15 , wherein, to correlate the location of the activating object with one of the first and second distinct sides, the instructions, when executed by at least one processor, further cause the processor to perform at least one operation comprising:
determining whether the activating object is performing a predetermined gesture and is less than a predetermined distance from a display device associated with the activatable object.
29. A touch-free gesture recognition method, comprising:
displaying a virtual 3D image that includes at least one activatable object, the activatable object having at least two distinct sides that are simultaneously activatable, wherein a first distinct side of the at least two distinct sides is associated with a first action and a second distinct side of the at least two distinct sides is associated with a second action;
receiving, from at least one image sensor, at least one image including a depiction of an activating object associated with a user;
correlating a location of the activating object with a location of one of the first and second distinct sides; and
if the location of the activating object correlates to the first distinct side, implementing the first action, and
if the location of the activating object correlates to the second distinct side, implementing the second action.
30. The touch-free gesture recognition method of claim 29 , wherein the first and second distinct virtual sides are each associated with at least one of a left side of the activatable object, a right side of the activatable object, a bottom side of the activatable object, and a top side of the activatable object.