Determining and utilizing corrections to robot actions
View Patent ↗Methods, apparatus, and computer-readable media for determining and utilizing human corrections to robot actions. In some implementations, in response to determining a human correction of a robot action, a correction instance is generated that includes sensor data, captured by one or more sensors of the robot, that is relevant to the corrected action. The correction instance can further include determined incorrect parameter(s) utilized in performing the robot action and/or correction information that is based on the human correction. The correction instance can be utilized to generate training example(s) for training one or model(s), such as neural network model(s), corresponding to those used in determining the incorrect parameter(s). In various implementations, the training is based on correction instances from multiple robots. After a revised version of a model is generated, the revised version can thereafter be utilized by one or more of the multiple robots.
1. A method implemented by one or more processors, comprising:
applying sensor data as input to a current model for a robot, the sensor data generated based on one or more sensors of the robot;
generating output over the current model based on the sensor data;
determining, based on the generated output, one or more parameters for a robot action directed toward an object in an environment of the robot, wherein the one or more parameters indicate a classification of the object;
receiving, during or after performance of the robot action by the robot based on the one or more parameters, user interface input that corrects the robot action;
initiating performance of the robot action based on the one or more parameters for the robot action, and based on one or more additional parameters that are determined using one or more additional models, for the robot, that are disparate from the current model;
in response to receiving the user interface input that corrects the robot action:
generating a correction instance that includes:
multiple instances of sensor data that are related to the robot action, and
transmitting the correction instance to one or more remote computing devices via a network interface;
receiving, subsequent to transmitting the correction instance, a revised version of the current model, wherein the revised version of the current model is trained based on:
the correction instance, and
additional correction instances from additional robots, the additional correction instances based on multiple instances of additional sensor data that are related to additional robot actions performed by each of the additional robots; and
in response to receiving the revised version of the current model, using the revised version of the current model in lieu of the current model in determining additional parameters for an additional robot action of the robot.
2. The method of claim 1 , wherein the multiple instances of the sensor data included in the correction instance include multiple instances of vision sensor data, and wherein generating the correction instance comprises:
including a first instance of the vision sensor data, in the correction instance, based on determining that the first instance captures the object at a first time from a first pose, and
including a second instance of the vision sensor data, in the correction instance, based on determining that the second instance captures the object at a second time from a second pose.
3. The method of claim 2 , wherein generating the correction instance comprises:
determining that the multiple instances of the vision sensor data are related to the robot action based on determining that the multiple instances include an identifier of the object.
4. The method of claim 2 , wherein the multiple instances of the vision sensor data are included in the sensor data applied to the current model in determining the one or more parameters for the robot action.
5. The method of claim 1 , wherein generating the correction instance comprises generating the correction instance to further include correction information that is based on the user interface input and that specifies an alternate classification for the object.
6. The method of claim 1 , wherein generating the correction instance comprises generating the correction instance to further include:
an indication of the current model based on the current model being used in determining the one or more parameters for the robot action.
7. The method of claim 1 , wherein generating the correction instance comprises generating the correction instance to further include:
the one or more parameters for the robot action.
8. The method of claim 1 , wherein the user interface input is natural language input.
9. A method implemented by one or more processors, comprising:
receiving user interface input during movement of a robot to perform an action directed toward an object;
determining that the user interface input indicates a correction to the action directed toward the object;
in response to determining that the user interface input indicates the correction:
generating a correction instance that includes:
multiple instances of vision sensor data that each captures the object, the vision sensor data generated by a vision sensor of the robot, and the multiple instances including a first instance that captures the object at a first time from a first pose, and a second instance that captures the object at a second time from a second pose;
wherein the multiple instances of the vision sensor data are included in the correction instance based on determining that each of the multiple instances captures the object; and
transmitting the correction instance to one or more remote computing devices via a network interface, wherein the one or more remote computing devices use the correction instance to generate a plurality of training examples for training a model.
10. The method of claim 9 , wherein the model is a neural network model and further comprising:
receiving, via the network interface, the neural network model after the neural network model is trained based on:
the plurality of training examples, and
additional training examples from additional correction instances from additional robots.
11. The method of claim 9 , wherein the correction instance further includes correction information that is based on the user interface input.
12. The method of claim 9 , further comprising:
determining a predicted classification of the object based on a local model that is local to the robot, wherein the action performed by the robot is performed based on the predicted classification;
wherein generating the correction instance includes generating the correction instance to further include the predicted classification of the object.
13. The method of claim 12 , further comprising:
generating the plurality of training examples, wherein the training examples include negative training examples that include training example output that indicates the predicted classification is not a correct classification.
14. The method of claim 9 , further comprising:
receiving initial user interface input;
determining that the initial user interface input indicates the object; and
generating robot control commands to perform the action directed toward the object in response to determining that the initial user interface input indicates the object.
15. The method of claim 14 , wherein the correction instance further includes correction information that is based on the initial user interface input.
16. The method of claim 9 , further comprising, in response to determining that the user interface input indicates the correction:
providing user interface output that requests additional correction details;
receiving, in response to the providing, further user interface input that includes the additional correction details; and
including the additional correction details in the correction instance.
17. The method of claim 9 , further comprising, in response to determining that the user interface input indicates the correction:
adjusting, based on the user interface input, a locally stored parameter associated with the object.
18. A method implemented by one or more processors, comprising:
determining a classification of an object in an environment of a robot;
receiving initial user interface input provided by a user;
determining that the initial user interface input conforms to the classification of the object;
in response to determining that the initial user interface input conforms to the classification of the object:
providing control commands to one or more actuators of the robot to perform an action directed toward the object;
receiving additional user interface input during performance of the action directed toward the object;
determining that the additional user interface input indicates a conflicting classification of the object that does not conform to the determined classification of the object;
in response to determining that the additional user interface input indicates a conflicting classification of the object:
generating a correction instance that includes the conflicting classification indicated by the additional user interface input, and
transmitting the correction instance to one or more remote computing devices via a network interface, wherein the one or more remote computing devices use the correction instance to generate a plurality of training examples for training a model.
19. The method of claim 18 , wherein the correction instance includes at least one instance of vision sensor data that captures the object, the vision sensor data generated by a vision sensor of the robot.
20. The method of claim 18 , wherein the correction instance includes multiple instances of vision sensor data that each captures the object, the vision sensor data generated by a vision sensor of the robot, and the multiple instances including a first instance that captures the object at a first time from a first pose, and a second instance that captures the object at a second time from a second pose.