Systems and methods for relative representation of spatial objects and disambiguation in an interface
Implementations described and claimed herein provide systems and methods for interaction between a user and a machine. In one implementation, a system is provided that receives an input from a user of a mobile machine which indicates or describes an object in the world. In one example, the user may gesture to the object which is detected by a visual sensor. In another example, the user may verbally describe the object which is detected by an audio sensor. The system receiving the input may then determine which object near the location of the user that the user is indicating. Such a determination may include utilizing known objects near the geographic location of the user or the autonomous or mobile machine.
1 . A method for interaction between a user and a device, the method comprising:
detecting a reference to an object type in an environment external to the device, the reference detected using an input system;
determining a position of an object in a three-dimensional space of the environment, the object corresponding to the reference;
determining a current spatial relationship between a user interface of the device and the position of the object by spatially orienting the object in the three-dimensional space of the environment relative to the user interface of the device according to a measured movement of the device through the environment; and
displaying a representation corresponding to the object, the representation displayed at a display location according to the current spatial relationship between the user interface of the device and the position of the object, the display location of the representation dynamically adjusted according to the measured movement of the device through the environment.
2 . The method of claim 1 , further comprising:
identifying the object as a match to the reference by disambiguating a non-tactile selection of the object, the non-tactile selection obtained using the input system.
3 . The method of claim 2 , wherein the non-tactile selection corresponds to an object indicator selected from a plurality of object indicators, each of the plurality of object indicators identified based on the object type.
4 . The method of claim 3 , wherein each of the plurality of object indicators is further identified based on an estimated geographic location of the device.
5 . The method of claim 3 , wherein each of the plurality of object indicators is displayed at a corresponding display location according to the measured movement of the device through the environment.
6 . The method of claim 2 , wherein at least one of the reference or the non-tactile selection is verbal.
7 . The method of claim 2 , wherein at least one of the reference or the non-tactile selection is a gesture.
8 . The method of claim 1 , wherein the representation includes information about the object.
9 . The method of claim 1 , wherein the representation includes navigation to the object.
10 . One or more tangible non-transitory computer-readable storage media storing computer-executable instructions for performing a process on a device, the process comprising:
detecting a reference to an object type in an environment external to the device;
determining a position of an object in a three-dimensional space of the environment, the object corresponding to the reference;
determining a current spatial relationship between a user interface of the device and the position of the object by spatially orienting the object in the three-dimensional space of the environment relative to the user interface of the device according to a measured movement of the device through the environment;
generating a representation corresponding to the object; and
causing the representation to display at a display location according to the current spatial relationship between the user interface of the device and the position of the object, the display location of the representation dynamically adjusted according to the measured movement of the device through the environment.
11 . The one or more tangible non-transitory computer-readable storage media of claim 10 , further comprising:
identifying the object as a match to the reference by disambiguating a non-tactile selection of the object.
12 . The one or more tangible non-transitory computer-readable storage media of claim 11 , wherein the non-tactile selection corresponds to an object indicator selected from a plurality of object indicators, each of the plurality of object indicators identified based on the object type.
13 . The one or more tangible non-transitory computer-readable storage media of claim 11 , wherein each of the plurality of object indicators is further identified based on an estimated geographic location of the device.
14 . The one or more tangible non-transitory computer-readable storage media of claim 12 , wherein each of the plurality of object indicators is displayed at a corresponding display location according to the measured movement of the device through the environment.
15 . The one or more tangible non-transitory computer-readable storage media of claim 11 , wherein at least one of the reference or the non-tactile selection is verbal.
16 . The one or more tangible non-transitory computer-readable storage media of claim 11 , wherein the representation includes navigation to the object.
17 . The one or more tangible non-transitory computer-readable storage media of claim 10 , wherein the reference includes a verbal request corresponding to the object.
18 . A system for interaction between a user and a device, the system comprising:
an input device configured to detect a reference to an object type in an environment external to the device;
at least one processor configured to determine a position of an object in a three-dimensional space of the environment, a current spatial relationship between a user interface of the device and the position of the object being determined by spatially orienting the object in the three-dimensional space of the environment relative to the user interface of the device according to a measured movement of the device through the environment; and
a display configured to display a representation of the object at a display location according to the current spatial relationship between the user interface of the device and the object.
19 . The system of claim 18 , wherein the input system is configured to receive a non-tactile selection from the user, the at least one processor configured to identify the object by disambiguating the non-tactile selection.
20 . The system of claim 18 , wherein the non-tactile selection corresponds to an object indicator selected from a plurality of object indicators, each of the plurality of object indicators identified based on the object type.