Systems and methods for interfacing with a user
View Patent ↗Systems and methods communicate an intent of an autonomous vehicle externally. In one implementation, scan data of a field around a travel path of an autonomous vehicle is obtained. The scan data is captured using at least one sensor. An object in the field around the travel path is determined from the scan data. The object is determined to be mutable or immutable. A navigation condition associated with the object is determined based on whether the object is mutable or immutable. The navigation condition is correlated to a portion of the travel path. Control operation(s) of the autonomous vehicle is determined for the portion of the travel path in response to the navigation condition. A representation link between the control operation(s) of the autonomous vehicle and the object is generated. A representation of the field around the travel path is rendered and includes the representation link.
1 . A method of communicating an intent of a device, the method comprising:
obtaining sensor data in connection with the device operating in an environment external to the device, one or more portions of the device each moveable along a path from a first location towards a second location;
generating a perception of the environment external to the device and of operation of the device, the perception generated based on the sensor data;
determining a condition based on the perception, the condition estimated to affect a movement of at least one portion of the one or more portions of the device;
generating one or more decisions, the one or more decisions generated by the device in response to the condition;
generating a representation of the one or more decisions of the device, the representation of the one or more decisions of the device including a representation link providing a correlation with the condition estimated to affect the movement of the at least one portion of the device; and
presenting the representation of the one or more decisions of the device to one or more users of the device using a presentation system, presentation of the representation including an indication of the condition affecting the movement of the at least one portion of the device.
2 . The method of claim 1 , wherein generating the one or more decisions includes generating a motion plan for moving the one or more portions of the device.
3 . The method of claim 1 , wherein the condition includes one or more objects within the environment external to the device and estimated to affect the movement of the at least one portion of the device.
4 . The method of claim 3 , wherein the one or more objects have a movement path estimated to intersect with the path of the at least one portion of the device.
5 . The method of claim 1 , wherein the representation includes a visual representation.
6 . The method of claim 5 , wherein the visual representation communicates a relevance of one or more objects within the environment.
7 . The method of claim 1 , wherein the condition corresponds to an interaction with one or more persons in the environment external to the device.
8 . The method of claim 1 , wherein the one or more decisions include at least one of modifying a speed of the movement, modifying a type of the movement, or modifying a direction of the movement.
9 . The method of claim 1 , wherein the representation includes an audio representation.
10 . The method of claim 1 , wherein the representation link includes a tactile representation.
11 . One or more tangible non-transitory computer-readable storage media storing computer-executable instructions for performing a process on a device, the process comprising:
obtaining sensor data in connection with the device operating in an environment external to the device;
generating a perception of the environment external to the device and of operation of the device, the perception generated based on the sensor data;
determining a condition based on the perception, the condition estimated to affect the operation of the device;
generating one or more decisions, the one or more decisions generated by the device in response to the condition;
generating a representation of the one or more decisions of the device, the representation of the one or more decisions of the device including a representation link providing a correlation with the condition estimated to affect the operation of the device; and
rendering the representation of the one or more decisions of the device to present to one or more users of the device using a presentation system.
12 . The one or more tangible non-transitory computer-readable storage media of claim 11 , wherein one or more portions of the device is moveable.
13 . The one or more tangible non-transitory computer-readable storage media of claim 12 , wherein generating the one or more decisions includes generating a motion plan for moving the one or more portions of the device.
14 . The one or more tangible non-transitory computer-readable storage media of claim 12 , wherein the condition is estimated to affect movement of the one or more portions of the device.
15 . The one or more tangible non-transitory computer-readable storage media of claim 14 , wherein one or more objects in the environment is estimated to affect the movement of the one or more portions of the device.
16 . The one or more tangible non-transitory computer-readable storage media of claim 11 , wherein the condition includes an interaction with one or more persons in the environment external to the device.
17 . The one or more tangible non-transitory computer-readable storage media of claim 11 , wherein the one or more decisions include or controlling one or more subsystems of the device.
18 . The one or more tangible non-transitory computer-readable storage media of claim 11 , wherein the representation includes at least one of a visual representation, an audio representation, or a tactile representation of the condition estimated to affect the operation of the device.
19 . The one or more tangible non-transitory computer-readable storage media of claim 11 , wherein the representation includes at least one of a visual representation, an audio representation, or a tactile representation of the one or more decisions.
20 . A device comprising:
a sensor system including one or more sensors, the sensor system configured to capture sensor data in connection with the device operating in an environment external to the device;
a controller configured to generate one or more decisions in response to a condition based on a perception, the condition estimated to affect the operation of the device, the perception generated based on the sensor data and corresponding to the operation of the device; and
a presentation system configured to present a representation of the one or more decisions of the device to one or more users of the device, the representation of the one or more decisions of the device including a representation link providing a correlation with the condition estimated to affect the operation of the device.