Method and system for the detection and augmentation of tactile interactions in augmented reality
Program actions may be initiated after detection of a predefined gesture by a user with a real-world object. Users may interact with their physical environment in an augmented reality by detecting interactions with real objects using a combination of location and motion detection, material identification using wearable sensors, or both. Based on detected sensor data from user interaction with a real-world object, a predefined gesture may be identified and a program action associated with that target interaction for the real-world object may be executed. In some cases, the user experience may be enhanced by providing haptic feedback in response to tactile gestures and resulting events.
1. A method comprising:
based on a user location, retrieving for each of a plurality of real-world objects (i) associated information identifying a gesture associated with the respective object and (ii) associated information identifying a program action;
determining whether the user is proximate to at least a first object of the plurality of objects;
after retrieving the information identifying the gesture associated with the first object, operating a sensor to detect a gesture of the user; and
in response to a determination that the user is proximate to the first object and that the gesture of the user corresponds to a first gesture associated with the first object, performing the program action associated with the first object.
2. The method of claim 1 , wherein determining whether the user is proximate to the first object comprises determining whether the first object is within reach of the user.
3. The method of claim 1 , wherein the operating of the sensor is performed in response to a determination that the user is proximate to at least one of the plurality of objects.
4. The method of claim 1 , wherein the first gesture is a gesture performed against a surface of the first object.
5. The method of claim 1 , wherein the information identifying a first gesture comprises a vibration profile, and wherein determining whether the gesture of the user corresponds to the first gesture comprises determining whether the gesture of the user matches the vibration profile.
6. The method of claim 5 , wherein the vibration profile is based on a texture of the first object.
7. The method of claim 1 , further comprising determining whether the user is proximate to a second object of the plurality of objects; and
in response to a determination that the user is proximate to both the first object and the second object, disambiguating the gesture of the user to determine whether to perform the program action associated with the first object or a program action associated with the second object.
8. The method of claim 1 , wherein the information retrieved for the first object comprises (i) associated information identifying a second gesture associated with the first object and (ii) associated information identifying a second program action associated with the first object.
9. The method of claim 1 , wherein determining whether the user is proximate to the first object comprises determining whether the user is closer to the first object than to a second object of the plurality of objects.
10. The method of claim 1 , wherein the sensor is a smart watch or a smart ring.
11. An apparatus comprising a processor and a non-transitory storage medium storing instructions operative, when executed on the processor, to perform functions including:
based on a user location, retrieving for each of a plurality of real-world objects (i) associated information identifying a gesture associated with the respective object and (ii) associated information identifying a program action;
determining whether the user is proximate to at least a first object of the plurality of objects;
after retrieving the information identifying the gesture associated with the first object, operating a sensor to detect a gesture of the user; and
in response to a determination that the user is proximate to the first object and that the gesture of the user corresponds to a first gesture associated with the first object, performing the program action associated with the first object.
12. The apparatus of claim 11 , wherein determining whether the user is proximate to the first object comprises determining whether the first object is within reach of the user.
13. The apparatus of claim 11 , wherein the operating of the sensor is performed in response to a determination that the user is proximate to at least one of the plurality of objects.
14. The apparatus of claim 11 , wherein the first gesture is a gesture performed against a surface of the first object.
15. The apparatus of claim 11 , wherein the information identifying a first gesture comprises a vibration profile, and wherein determining whether the gesture of the user corresponds to the first gesture comprises determining whether the gesture of the user matches the vibration profile.
16. The apparatus of claim 15 , wherein the vibration profile is based on a texture of the first object.
17. The apparatus of claim 11 , wherein the functions further include determining whether the user is proximate to a second object of the plurality of objects; and
in response to a determination that the user is proximate to both the first object and the second object, disambiguating the gesture of the user to determine whether to perform the program action associated with the first object or a program action associated with the second object.
18. The apparatus of claim 11 , wherein the information retrieved for the first object comprises (i) associated information identifying a second gesture associated with the first object and (ii) associated information identifying a second program action associated with the first object.
19. The apparatus of claim 11 , wherein determining whether the user is proximate to the first object comprises determining whether the user is closer to the first object than to a second object of the plurality of objects.
20. The apparatus of claim 11 , wherein the sensor is a smart watch or a smart ring.