Domain adaptation using simulation to simulation transfer
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a generator neural network to adapt input images.
1. A method, comprising:
receiving an input image of a real-word environment captured while a robotic agent is interacting with the real-world environment;
processing the input image of the real-world environment using a trained generator neural network to generate an adapted image from the input image, the trained generator neural network having been trained to adapt images of a randomized simulation of the real-world environment into images of a canonical simulation of the real-world environment;
providing the adapted image as input to a control policy for the robotic agent to select a subsequent action to be performed by the robotic agent; and
controlling the robotic agent to perform the selected subsequent action in the real-world environment.
2. The method of claim 1 , wherein the control policy for the robotic agent is trained to control the robotic agent at least in part in a simulated version of the real-world environment.
3. The method of claim 2 , wherein the trained generator neural network is trained with canonical simulated training images captured during the training of the control policy in the simulated version of the real-world environment.
4. The method of claim 1 , wherein the trained generator neural network is trained using a plurality of simulation training inputs, each simulation training input comprising (i) a canonical simulated training image of the canonical simulation of the real-world environment and (ii) a corresponding randomized simulated training image of the randomized simulation of the real-world environment.
5. The method of claim 4 , wherein the corresponding randomized simulated training image of each simulation training input is generated by randomizing one or more characteristics of a scene depicted in the canonical simulated training image.
6. The method of claim 4 , wherein the trained generator neural network is trained to optimize an objective function that includes one or more terms that encourage adapted images generated by the generator neural network by processing randomized simulated training images to be similar to corresponding canonical simulated training images.
7. The method of claim 6 , wherein the trained generator neural network is trained jointly with a canonical-randomized discriminator neural network having a plurality of canonical-randomized discriminator parameters,
wherein the canonical-randomized discriminator neural network is configured to process input images in accordance with the canonical-randomized discriminator parameters to classify each input image as either being an adapted image or a canonical simulated image, and
wherein the objective function includes a term that penalizes the generator neural network for generating training adapted images that are accurately classified by the canonical-randomized discriminator neural network.
8. The method of claim 7 , wherein the trained generator neural network is trained by repeatedly alternating between the following:
performing an iteration of a machine learning training technique that adjusts generator parameters to minimize the term by determining gradients with respect to the generator parameters for a batch of simulation training inputs; and
performing an iteration of a machine learning training technique that adjusts the canonical-randomized discriminator parameters to maximize the term by determining gradients with respect to the canonical-randomized discriminator parameters for a batch of simulation training inputs.
9. The method of claim 1 , wherein the trained generator neural network is trained using an objective function that includes a term that encourages visual similarity between adapted training images generated by the generator neural network and corresponding randomized simulation images.
10. The method of claim 1 , wherein the trained generator neural network is trained using an objective function that includes a term that encourages semantic similarity between predicted segmentation masks generated by the generator neural network and corresponding ground truth segmentation masks.
11. The method of claim 1 , wherein the trained generator neural network is trained using an objective function that includes a term that encourages similarity between predicted depth maps generated by the generator neural network and corresponding ground truth depth maps.
12. The method of claim 1 , wherein the trained generator neural network is trained using an objective function that includes a term that encourages adapted images generated by the generator neural network by processing real-world training images to appear to be images of the canonical simulation while maintaining semantics of the real-world training images.
13. The method of claim 1 , wherein the trained generator neural network is trained using canonical training images captured while a simulated version of the robotic agent interacts with a simulated version of the real-world environment.
14. The method of claim 13 , wherein the canonical training images are captured during the training of the control policy for the robotic agent in the simulated version of the real-world environment.
15. The method of claim 1 , wherein the images of the randomized simulation of the real-world environment have randomized lighting characteristics of a scene.
16. The method of claim 1 , wherein the images of the randomized simulation of the real-world environment have randomized texture characteristics of a scene.
17. The method of claim 1 , wherein the images of the randomized simulation of the real-world environment have randomized object properties of objects in a scene.
18. A robotic agent configured to:
receive an input image of a real-word environment captured while the robotic agent is interacting with the real-world environment;
process the input image of the real-world environment using a trained generator neural network to generate an adapted image from the input image, the trained generator neural network having been trained to adapt images of a randomized simulation of the real-world environment into images of a canonical simulation of the real-world environment;
provide the adapted image as input to a control policy for the robotic agent to select a subsequent action to be performed by the robotic agent; and
control the robotic agent to perform the selected subsequent action in the real-world environment.
19. The robotic agent of claim 18 , wherein the control policy for the robotic agent is trained to control the robotic agent at least in part in a simulated version of the real-world environment.
20. One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
receiving an input image of a real-word environment captured while a robotic agent is interacting with the real-world environment;
processing the input image of the real-world environment using a trained generator neural network to generate an adapted image from the input image, the trained generator neural network having been trained to adapt images of a randomized simulation of the real-world environment into images of a canonical simulation of the real-world environment;
providing the adapted image as input to a control policy for the robotic agent to select a subsequent action to be performed by the robotic agent; and
controlling the robotic agent to perform the selected subsequent action in the real-world environment.