Machine learning methods and apparatus for automated robotic placement of secured object in appropriate location

1. A method implemented by one or more processors of a robot, comprising:

identifying one or more object images, the one or more object images each being captured by a vision component associated with the robot, and the one or more object images each capturing an object in an environment of the robot;

generating a target placement input that indicates at least one characteristic of a placement location for the object, wherein generating the target placement input is based on classification output generated by processing the one or more object images using a trained classification model;

while an end effector of the robot is securing the object:

identifying a current image captured by the vision component, the current image capturing the end effector and at least part of the environment;

generating one or more candidate end effector actions each defining corresponding motion to move the end effector;

for each candidate end effector action of the one or more candidate end effector actions:

processing, using a trained placement model, the target placement input, the current image, and the candidate end effector action, and

generating, based on the processing, a prediction of successful placement of the object with application of the motion defined by the corresponding candidate end effector action;

generating control commands that conform to the candidate end effector action that, when processed using the trained placement model, resulted in the prediction that is most indicative of successful placement of the object; and

providing the control commands to one or more actuators of the robot to cause the end effector to move while continuing to secure the object;

determining to cause the end effector to release the object after providing the control commands; and

in response to determining to cause the end effector to release the object, providing one or more additional control commands to cause the end effector to release the object.

2. The method of claim 1 , wherein determining to cause the end effector to release the object is based on determining that the prediction, that is most indicative of successful placement of the object, satisfies one or more criteria.

3. The method of claim 1 , wherein determining to cause the end effector to release the object comprises:

generating an additional prediction based on processing, using the trained placement model: the target placement input, an additional current image, and a null end effector action, wherein the additional current image is captured after providing the end effector motion command; and

determining to cause the end effector to release the object based on the additional prediction satisfying one or more criteria.

4. The method of claim 1 , wherein the one or more object images are captured while the object is secured by the end effector of the robot.

5. The method of claim 1 , further comprising:

causing, after the end effector has secured the object, the end effector of the robot to move to one or more predetermined poses relative to the vision component; and

causing the one or more object images to be captured while the end effector is in the one or more predetermined poses;

wherein identifying the one or more object images is based on the one or more object images being captured while the end effector is in the one or more predetermined poses.

6. The method of claim 1 , wherein the classification output indicates at least one semantic classification of the object, and wherein generating the target placement input is based on the semantic classification being mapped to the target placement input.

7. The method of claim 6 , wherein the target placement input comprises a semantic identifier that identifies one of a plurality of candidate placement locations.

8. The method of claim 7 , wherein the plurality of candidate placement locations comprise two or more of a trash bin, a recycle bin, and a compost bin.

9. The method of claim 6 , wherein the semantic classification of the object indicates one or more of: a class of the object, a weight of the object, a size of the object, and a material of the object.

10. The method of claim 1 , wherein the target placement input comprises an image that is similar to the placement location.

11. The method of claim 1 , wherein the target placement input comprises an image-based spatial identifier of the target placement location, and wherein generating the target placement input comprises:

processing an additional image that is of the environment and that is captured by the vision component;

determining, based on the classification output, a portion of the additional image that conforms to the target placement location; and

generating the image-based spatial identifier based on the portion of the additional image that conforms to the target placement location.

12. The method of claim 1 , further comprising:

identifying a task being performed by the robot;

wherein generating the target placement input is further based on the task being performed by the robot.

13. A method implemented by one or more processors of a robot, comprising:

generating a target placement input that indicates at least one characteristic of a placement location for an object secured by an end effector of a robot;

at each of a plurality of iterations while the end effector of the robot is securing the object, and until determining that at least one release criteria is satisfied:

moving the end effector of the robot while the end effector of the robot is securing the object, wherein moving the end effector of the robot is based on predictions of successful placement that are each generated based on processing, using a trained placement model, of:

the target placement input,

a corresponding current image that is captured by a vision component of the robot and that captures the end effector, and

a corresponding candidate end effector action that defines corresponding motion to move the end effector;

in response to determining that the at least one release criteria is satisfied:

causing the end effector to release the secured object.

14. The method of claim 13 , wherein determining that the at least one criteria is satisfied is based on at least one of:

one of the generated predictions, or

an additional prediction generated based on processing, using the trained placement model, of: the target placement input, an additional current image, and a null end effector action.

15. The method of claim 13 , further comprising:

identifying a task being performed by the robot;

wherein generating the target placement input is based on the task being performed by the robot.

16. The method of claim 13 , further comprising:

classifying the object based on one or more images of the object that are captured by the vision component of the robot;

wherein generating the target placement input is based on the classifying.

17. A method of training a placement machine learning model, the method implemented by one or more processors and comprising:

identifying, by one or more processors, a plurality of training examples generated based on sensor output from one or more robots during a plurality of placement attempts by the robots, each of the placement attempts being performed over a plurality of instances of time,

each of the training examples including training example input comprising:

a corresponding image for a corresponding instance of time of a corresponding placement attempt of the placement attempts, the corresponding image capturing a robotic end effector at the corresponding instance of time while the corresponding end effector is securing a corresponding object, and at least one corresponding placement location,

a corresponding end effector action defining motion of the end effector to move from a corresponding instance of time pose of the end effector at the corresponding instance of time to a corresponding final pose of the end effector for the corresponding placement attempt, and

a corresponding target placement input that indicates at least one characteristic of a corresponding target placement location for the object;

each of the training examples including training example output comprising:

a corresponding placement success label that indicates whether the corresponding object was placed in the target placement location as a result of the corresponding placement attempt;

training, by one or more of the processors, the placement machine learning based on applying the training example input of the training examples to the placement machine learning model and updating parameters of the placement machine learning based on the training example output of the training examples.

18. The method of claim 16 , wherein the robots comprise simulated robots simulated by one or more robot simulators, and wherein the placement attempts comprises simulated placement attempts performed using the one or more robot simulators.