Visual annotations in robot control interfaces
Methods, apparatus, systems, and computer-readable media are provided for visually annotating rendered multi-dimensional representations of robot environments. In various implementations, an entity may be identified that is present with a telepresence robot in an environment. A measure of potential interest of a user in the entity may be calculated based on a record of one or more interactions between the user and one or more computing devices. In some implementations, the one or more interactions may be for purposes other than directly operating the telepresence robot. In various implementations, a multi-dimensional representation of the environment may be rendered as part of a graphical user interface operable by the user to control the telepresence robot. In various implementations, a visual annotation may be selectively rendered within the multi-dimensional representation of the environment in association with the entity based on the measure of potential interest.
1. A method implemented by one or more processors, comprising:
identifying an object that is present in an environment in which a robot operates or will operate;
determining one or more tasks that are performable by the robot to physically interact with the object using one or more end effectors of the robot;
rendering, as part of a graphical user interface operable by an individual to control or monitor the robot, a visual representation of the environment; and
selectively rendering a visual annotation with the one or more tasks within the visual representation of the environment;
wherein the object comprises a vehicle, and the one or more tasks include one or more of filling the vehicle with gas, inflating one or more tires of the vehicle, or washing the vehicle.
2. The method of claim 1 , wherein the identifying is based on data captured by one or more vision sensors integral with the robot.
3. The method of claim 1 , wherein the one or more tasks that are performable by the robot are determined from a library of tasks that is populated based on past robot interactions with objects.
4. The method of claim 1 , wherein the visual annotation is selectable by the individual to cause the robot to perform one or more of the tasks.
5. The method of claim 1 , wherein the visual representation of the environment comprises a live video feed provided by the robot, and wherein selectively rendering the visual annotation comprises overlaying the visual annotation on the live video feed.
6. The method of claim 1 , wherein the visual representation of the environment comprises a live map view of the environment, and wherein selectively rendering the visual annotation comprises overlaying the visual annotation on the live map view.
7. At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by a computing device, cause the computing device to perform the following operations:
identifying an object that is present in an environment in which a robot operates or will operate;
determining one or more tasks that are performable by the robot to physically interact with the object using one or more end effectors of the robot;
rendering, as part of a graphical user interface operable by an individual to control or monitor the robot, a visual representation of the environment; and
selectively rendering a visual annotation with the one or more tasks within the visual representation of the environment, wherein the object comprises a vehicle, and the one or more tasks include one or more of filling the vehicle with gas, inflating one or more tires of the vehicle, or washing the vehicle.
8. The non-transitory computer-readable medium of claim 7 , wherein the identifying is based on data captured by one or more vision sensors integral with the robot.
9. The non-transitory computer-readable medium of claim 7 , wherein the one or more tasks that are performable by the robot are determined from a library of tasks that is populated based on past robot interactions with objects.
10. The non-transitory computer-readable medium of claim 7 , wherein the visual annotation is selectable by the individual to cause the robot to perform one or more of the tasks.
11. A system including memory and one or more processors operable to execute instructions stored in the memory, comprising instructions to perform the following operations:
identifying an object that is present in an environment in which a robot operates or will operate;
determining one or more tasks that are performable by the robot to physically interact with the object using one or more end effectors of the robot;
rendering, as part of a graphical user interface operable by an individual to control or monitor the robot, a visual representation of the environment; and
selectively rendering a visual annotation with the one or more tasks within the visual representation of the environment, wherein the object comprises a vehicle, and the one or more tasks include one or more of filling the vehicle with gas, inflating one or more tires of the vehicle, or washing the vehicle.
12. The system of claim 11 , wherein the identifying is based on data captured by one or more vision sensors integral with the robot.
13. The system of claim 11 , wherein the one or more tasks that are performable by the robot are determined from a library of tasks that is populated based on past robot interactions with objects.
14. The system of claim 11 , wherein the visual annotation is selectable by the individual to cause the robot to perform one or more of the tasks.
15. The system of claim 11 , wherein the visual representation of the environment comprises a live video feed provided by the robot, and wherein selectively rendering the visual annotation comprises overlaying the visual annotation on the live video feed.
16. The system of claim 11 , wherein the visual representation of the environment comprises a live map view of the environment, and wherein selectively rendering the visual annotation comprises overlaying the visual annotation on the live map view.