Object-Based Robot Control

1 . A computer-implemented method comprising:

receiving, by data processing hardware of a robot, first sensor data from one or more sensors of the robot, the first sensor data corresponding to an environment of the robot;

determining, by the data processing hardware, based on the first sensor data, a first object in the environment and a second object in the environment;

identifying, by the data processing hardware, a first action from a first set of actions associated with the first object;

identifying, by the data processing hardware, a second action from a second set of actions associated with the second object;

generating, by the data processing hardware, a mission to perform the first action and the second action; and

instructing, by the data processing hardware, performance of the mission by the robot.

2 . The computer-implemented method of claim 1 , further comprising:

generating a graphical representation of the first set of actions; and

instructing display of the graphical representation based on a location of the robot corresponding to a location of the first object.

3 . The computer-implemented method of claim 1 , further comprising:

generating a graphical representation based on performance of the first action; and

instructing display of the graphical representation.

4 . The computer-implemented method of claim 1 , further comprising:

receiving an input indicating selection of the first action, wherein

identifying the first action is based on the input.

5 . The computer-implemented method of claim 1 , further comprising:

generating a graphical representation based on the environment, wherein

the graphical representation indicates the first set of actions and the first object; and

receiving, via an interaction with the graphical representation, an input indicating selection of the first action, wherein identifying the first action is based on the input.

6 . The computer-implemented method of claim 1 , further comprising:

generating a graphical representation based on the environment, wherein the graphical representation indicates the first object; and

receiving, via an interaction with the graphical representation, an input defining the first set of actions.

7 . The computer-implemented method of claim 1 , wherein instructing performance of the mission by the robot comprises:

instructing performance of the first action by the robot.

8 . The computer-implemented method of claim 1 , wherein instructing performance of the mission by the robot comprises:

instructing performance of the first action by the robot based on a location of the robot corresponding to a location of the first object.

9 . The computer-implemented method of claim 1 , wherein generating the mission comprises:

scheduling the robot to perform the first action at a particular time period.

10 . The computer-implemented method of claim 1 , wherein the mission indicates an order of performance of the first action and performance of the second action.

11 . The computer-implemented method of claim 1 , wherein the first set of actions comprises a set of actions to interact with the first object.

12 . The computer-implemented method of claim 1 , wherein the first set of actions are different from the second set of actions.

13 . The computer-implemented method of claim 1 , wherein the first object and the second object are a same type of object.

14 . The computer-implemented method of claim 1 , further comprising:

associating the first set of actions with the first object based on second sensor data.

15 . The computer-implemented method of claim 1 , further comprising:

matching the first object to a respective object of a set of objects; and

identifying the first set of actions based on the set of objects, wherein the set of objects associates the first object with the first set of actions.

16 . A system comprising:

data processing hardware; and

memory in communication with the data processing hardware, the memory storing instructions, wherein execution of the instructions by the data processing hardware causes the data processing hardware to:

receive first sensor data from one or more sensors of a robot, the first sensor data corresponding to an environment of the robot;

determine, based on the first sensor data, a first object in the environment and a second object in the environment;

identify a first action from a first set of actions associated with the first object;

identify a second action from a second set of actions associated with the second object;

generate a mission to perform the first action and the second action; and

instruct performance of the mission by the robot.

17 . The system of claim 16 , wherein the execution of the instructions by the data processing hardware further causes the data processing hardware to:

identify the first set of actions using a machine learning model.

18 . The system of claim 16 , wherein to determine the first object and the second object, the execution of the instructions by the data processing hardware further causes the data processing hardware to:

identify the first object using an object detection model.

19 . A robot comprising:

one or more sensors;

data processing hardware; and

memory in communication with the data processing hardware, the memory storing instructions, wherein execution of the instructions by the data processing hardware causes the data processing hardware to:

receive first sensor data from the one or more sensors, the first sensor data corresponding to an environment of the robot;

determine, based on the first sensor data, a first object in the environment and a second object in the environment;

identify a first action from a first set of actions associated with the first object;

identify a second action from a second set of actions associated with the second object;

generate a mission to perform the first action and the second action; and

instruct performance of the mission.

20 . The robot of claim 19 , wherein the first action comprises a navigational behavior of the robot.