Domain adaptation using simulation to simulation transfer

1. A method of training a generator neural network having a plurality of generator parameters to process input images of scenes in a real-world environment to generate a respective adapted image for each input image, wherein the respective adapted image for each input image appears to be an image of the same scene as in the input image but in a canonical simulation of the real-world environment, and wherein the method comprises:

obtaining a plurality of simulation training inputs, each simulation training input comprising (i) a canonical simulated training image of the canonical simulation of the real-world environment and (ii) a corresponding randomized simulated training image of a randomized simulation of the real-world environment generated by randomizing one or more characteristics of a scene depicted in the canonical simulated training image; and

training the generator neural network on training data comprising the simulation training inputs to determine trained values of the generator parameters from initial values of the generator parameters, comprising:

optimizing an objective function that includes one or more terms that encourage adapted images generated by the generator neural network by processing the randomized simulated training images to be similar to the corresponding canonical simulated training images.

2. The method of claim 1 , wherein:

training the generator neural network comprises training the generator neural network jointly with a canonical-randomized discriminator neural network having a plurality of canonical-randomized discriminator parameters,

the canonical-randomized discriminator neural network is configured to process input images in accordance with the canonical-randomized discriminator parameters to classify each input image as either being an adapted image or a canonical simulated image, and

the objective function includes a first term that (i) penalizes the generator neural network for generating training adapted images that are accurately classified by the canonical-randomized discriminator neural network and (ii) encourages high classification accuracy of the canonical-randomized discriminator neural network on the canonical and randomized simulated training images.

3. The method of claim 2 , wherein training the generator neural network comprises repeatedly alternating between the following:

performing an iteration of a machine learning training technique that adjusts the generator parameters to minimize the first term by determining gradients with respect to the generator parameters for a batch of simulation training inputs; and

performing an iteration of a machine learning training technique that adjusts the canonical-randomized discriminator parameters to maximize the first term by determining gradients with respect to the canonical-randomized discriminator parameters for a batch of simulation training inputs.

4. The method of claim 1 , wherein the objective function includes a third term that encourages visual similarity between the adapted training images generated by the generator neural network and the corresponding randomized simulation images.

5. The method of claim 1 ,

wherein each simulation training input further comprises a ground truth segmentation mask of the canonical simulated training image in the simulation training input,

wherein the generator neural network is further configured to generate a predicted segmentation mask of each input image, and

wherein the objective function includes a fourth term that encourages semantic similarity between the predicted segmentation masks generated by the generator neural network and the corresponding ground truth segmentation masks.

6. The method of claim 1 ,

wherein each simulation training input further comprises a ground truth depth map of the canonical simulated training image in the simulation training input,

wherein the generator neural network is further configured to generate a predicted depth map of each input image, and

wherein the objective function includes a fifth term that encourages similarity between the predicted depth maps generated by the generator neural network and the corresponding ground truth depth maps.

7. The method of claim 1 , further comprising:

obtaining a plurality of real-world training images, each real-world training image being an image of the real-world environment, wherein:

the training data further comprises the plurality of real-world training images, and

the objective function includes one or more terms that encourage adapted images generated by the generator neural network by processing the real-world training images to appear to be images of the canonical simulation while maintaining the semantics of the corresponding real-world training images.

8. The method of claim 1 , wherein the canonical training images are images captured while a simulated version of a robotic agent interacts with a simulated version of the real-world environment.

9. The method of claim 8 , wherein the canonical training images are captured during the training of a control policy for the robotic agent in the simulated version of the real-world environment.

10. The method of claim 1 , further comprising, after the generator neural network is trained, using the trained generator neural network in controlling the robotic agent interacting with the real-world environment to cause the robotic agent to perform a robotic task, comprising:

receiving an input image of the real-word environment captured while the robotic agent is interacting with the real-world environment;

processing the input image of the real-world environment using the trained generator neural network to generate an adapted image from the input image; and

providing the adapted image as input to a control policy for the robotic agent to select a subsequent action to be performed by the robotic agent.

11. The method of claim 10 , wherein the control policy for the robotic agent is trained to control the robotic agent at least in part in a simulated version of the real-world environment.

12. The method of claim 11 , wherein obtaining a plurality of simulation training inputs comprises:

capturing the canonical simulated training images in the plurality of simulation training inputs during the training of the control policy in the simulated version of the environment.

13. The method of claim 1 , wherein randomizing one or more characteristics of a scene depicted in the canonical simulated training image comprises randomizing one or more of:

lighting characteristics of the scene, texture characteristics of the scene, or object properties of objects in the scene.

14. A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations training a generator neural network having a plurality of generator parameters to process input images of scenes in a real-world environment to generate a respective adapted image for each input image, wherein the respective adapted image for each input image appears to be an image of the same scene as in the input image but in a canonical simulation of the real-world environment, and wherein the operations comprise:

obtaining a plurality of simulation training inputs, each simulation training input comprising (i) a canonical simulated training image of the canonical simulation of the real-world environment and (ii) a corresponding randomized simulated training image of a randomized simulation of the real-world environment generated by randomizing one or more characteristics of a scene depicted in the canonical simulated training image; and

training the generator neural network on training data comprising the simulation training inputs to determine trained values of the generator parameters from initial values of the generator parameters, comprising:

optimizing an objective function that includes one or more terms that encourage adapted images generated by the generator neural network by processing the randomized simulated training images to be similar to the corresponding canonical simulated training images.

15. The system of claim 14 , wherein: training the generator neural network comprises training the generator neural network jointly with a canonical-randomized discriminator neural network having a plurality of canonical-randomized discriminator parameters, the canonical-randomized discriminator neural network is configured to process input images in accordance with the canonical-randomized discriminator parameters to classify each input image as either being an adapted image or a canonical simulated image, and the objective function includes a first term that (i) penalizes the generator neural network for generating training adapted images that are accurately classified by the canonical-randomized discriminator neural network and (ii) encourages high classification accuracy of the canonical-randomized discriminator neural network on the canonical and randomized simulated training images.

16. The system of claim 15 , wherein training the generator neural network comprises repeatedly alternating between the following:

performing an iteration of a machine learning training technique that adjusts the generator parameters to minimize the first term by determining gradients with respect to the generator parameters for a batch of simulation training inputs; and

performing an iteration of a machine learning training technique that adjusts the canonical-randomized discriminator parameters to maximize the first term by determining gradients with respect to the canonical-randomized discriminator parameters for a batch of simulation training inputs.

17. The system of claim 14 , wherein the objective function includes a third term that encourages visual similarity between the adapted training images generated by the generator neural network and the corresponding randomized simulation images.

18. The system of claim 14 ,

wherein each simulation training input further comprises a ground truth segmentation mask of the canonical simulated training image in the simulation training input,

wherein the generator neural network is further configured to generate a predicted segmentation mask of each input image, and

wherein the objective function includes a fourth term that encourages semantic similarity between the predicted segmentation masks generated by the generator neural network and the corresponding ground truth segmentation masks.

19. The system of claim 14 ,

wherein each simulation training input further comprises a ground truth depth map of the canonical simulated training image in the simulation training input,

wherein the generator neural network is further configured to generate a predicted depth map of each input image, and

wherein the objective function includes a fifth term that encourages similarity between the predicted depth maps generated by the generator neural network and the corresponding ground truth depth maps.

20. One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

training a generator neural network having a plurality of generator parameters to process input images of scenes in a real-world environment to generate a respective adapted image for each input image, wherein the respective adapted image for each input image appears to be an image of the same scene as in the input image but in a canonical simulation of the real-world environment, and wherein the operations comprise:

obtaining a plurality of simulation training inputs, each simulation training input comprising (i) a canonical simulated training image of the canonical simulation of the real-world environment and (ii) a corresponding randomized simulated training image of a randomized simulation of the real-world environment generated by randomizing one or more characteristics of a scene depicted in the canonical simulated training image; and

training the generator neural network on training data comprising the simulation training inputs to determine trained values of the generator parameters from initial values of the generator parameters, comprising:

optimizing an objective function that includes one or more terms that encourage adapted images generated by the generator neural network by processing the randomized simulated training images to be similar to the corresponding canonical simulated training images.