Generating a model for an object encountered by a robot

1. A method, comprising:

receiving vision sensor data generated by a vision sensor associated with a robot, the vision sensor data capturing an object in an environment of the robot from multiple vantages, and the vision sensor data comprising two-dimensional images or two-dimensional images with depth;

generating a full or partial three-dimensional object model of the object based on the vision sensor data;

generating a plurality of rendered images based on the three-dimensional object model, wherein the rendered images capture the three-dimensional object model at a plurality of different poses relative to viewpoints of the rendered images, and wherein generating the rendered images based on the object model comprises:

rendering a first image that renders the object model and that includes first additional content; and

rendering a second image that renders the object model and that includes second additional content that is distinct from the first additional content:

generating training examples that each include a corresponding one of the rendered images as training example input and that each include an indication of the object as training example output; and

training a machine learning model based on the training examples.

2. The method of claim 1 , wherein rendering the first image with the first additional content further comprises rendering the object model onto a first background, and wherein rendering the second image with second additional content further comprises rendering the object model onto a second background that is distinct from the first background.

3. The method of claim 2 , further comprising:

selecting the first background and the second background based on the environment of the robot.

4. The method of claim 3 , wherein selecting the first background based on the environment of the robot comprises:

selecting the first background based on an additional image captured by the vision sensor or an additional vision sensor in the environment of the robot.

5. The method of claim 1 , further comprising:

selecting the first additional content based on the environment of the robot.

6. The method of claim 1 , wherein the training example output of each of the training examples further includes a corresponding pose of the object model in the corresponding one of the rendered images.

7. The method of claim 1 , wherein the rendered images each include a plurality of color channels and a depth channel.

8. The method of claim 1 , wherein the vision sensor data is received from the robot over one or more network interfaces and further comprising:

providing, via one or more of the network interfaces, the trained machine learning model to the robot for use by the robot.

9. A system comprising:

one or more processors executing instructions stored in memory to cause the one or more processors to:

receive vision sensor data generated by a vision sensor of a robot, the vision sensor data capturing an object from multiple vantages, and the vision sensor data comprising two-dimensional images or two-dimensional images with depth;

generate a full or partial three-dimensional object model of the object based on the vision sensor data;

generate a plurality of rendered images based on the three-dimensional object model, wherein the rendered images capture the three-dimensional object model at a plurality of different poses relative to viewpoints of the rendered images, and wherein in generating the rendered images based on the object model one or more of the processors are to:

render a first image that renders the object model and that includes first additional content; and

render a second image that renders the object model and that includes second additional content that is distinct from the first additional content;

generate training examples that each include a corresponding one of the rendered images as training example input and that each include an indication of the object as training example output; and

train a machine learning model based on the training examples.

10. The system of claim 9 , wherein in rendering the first image with the first additional content one or more of the processors are to render the object model onto a first background, and wherein in rendering the second image with second additional content one or more of the processors are to render the object model onto a second background that is distinct from the first background.

11. The system of claim 10 , wherein in executing the instructions one or more of the processor are further to:

select the first background and the second background based on the environment of the robot.

12. The system of claim 11 , wherein in selecting the first background based on the environment of the robot one or more of the processors are to:

select the first background based on an additional image captured by the vision sensor or an additional vision sensor in the environment of the robot.

13. The system of claim 9 , wherein in executing the instructions one or more of the processor are further to:

select the first additional content based on the environment of the robot.

14. The system of claim 9 , wherein the training example output of each of the training examples further includes a corresponding pose of the object model in the corresponding one of the rendered images.

15. The system of claim 9 , wherein the rendered images each include a plurality of color channels and a depth channel.