Training a simulation-to-real model, based on perception losses, to mitigate reality gap in robotics and/or other applications
Implementations disclosed herein relate to mitigating the reality gap through training a simulation-to-real machine learning model (“Sim2Real” model) based on loss(es) generated using already trained perception model(s), such as perception model(s) trained to process an image and generate predicted output(s) (e.g., predicted bounding area(s) and/or predicted classification(s) for object(s) in a given image). In some implementations, the perception model(s) can be trained to generate predicted output(s) based on processing real image(s), and can be used to process a predicted real image to generate the predicted output(s). The predicted output(s) can be compared to ground truth output(s), from the simulator, to generate a perception loss. The Sim2Real model can be updated based on the generated perception loss. In various implementations, the Sim2Real model can additionally and/or alternatively be updated based on additional loss(es). Once updated, the Sim2Real model can be utilized in generating robotic control policies to control a real robot.
1 . A method implemented by one or more processors, the method comprising:
processing a simulated image, using a simulation-to-real generator model of a generative adversarial network model, to generate a predicted real image that corresponds to the simulated image, wherein the simulated image is generated using a simulator;
processing the predicted real image, using a real perception model, to generate predicted output for the predicted real image,
wherein the real perception model is trained based on real training instances, each of the real training instances comprising: real training instance input that includes a corresponding real image, and real training instance output that includes corresponding ground truth output for the corresponding real image;
generating a perception loss based on comparing the predicted output for the predicted real image that is generated using the real perception model to ground truth output for the simulated image that is generated using the simulator in generating the simulated image;
updating, based on the generated perception loss, the simulation-to-real generator model that was used to generate the predicted real image and based on processing the simulated image; and
subsequent to updating the simulation-to-real generator model:
generating a robotic control policy for a real robot based on simulated training data generated using the simulation-to-real generator model; and
causing the real robot to be controlled based on the generated robotic control policy.
2 . The method of claim 1 , wherein the predicted output for the predicted real image that is generated using the real perception model includes a predicted bounding area corresponding to an object that is captured in the predicted real image.
3 . The method of claim 2 , wherein comparing the predicted output that is generated using the real perception model to the ground truth output for the simulated image that is generated using the simulator in generating the simulated image comprises:
comparing the predicted bounding area corresponding to the object that is captured in the predicted real image to a ground truth bounding area corresponding to the object captured in the simulated image.
4 . The method of claim 3 , wherein the ground truth bounding area is:
determined by the simulator when the simulated image is generated.
5 . The method of claim 3 , wherein comparing the predicted bounding area to the ground truth bounding area comprises one or more of:
comparing a size of the predicted bounding area to a ground truth size of the ground truth bounding area,
comparing a center point of the predicted bounding area to a ground truth center point of the ground truth bounding area, or
comparing an orientation of the predicted bounding area to a ground truth orientation of the ground truth bounding area.
6 . The method of claim 3 , wherein generating the perception loss based on comparing the predicted output for the predicted real image that is generated using the real perception model to the ground truth output for the simulated image that is generated using the simulator in generating the simulated image comprises:
determining a difference in N-dimensional space between the predicted bounding area and the ground truth bounding area, wherein the N-dimensional space is one or more of: a 1-dimensional space, 2-dimensional space, 2.5-dimensional space, or 3-dimensional space; and
generating the perception loss as a function of the difference in the N-dimensional space between the predicted bounding area and the ground truth bounding area.
7 . The method of claim 1 , wherein the predicted output for the predicted real image that is generated using the real perception model includes a predicted probability corresponding to an object classification of an object captured in the predicted real image.
8 . The method of claim 7 , wherein comparing the predicted output that is generated using the real perception model to the ground truth output for the simulated image that is generated using the simulator in generating the simulated image comprises:
comparing the predicted probability corresponding to the object classification of the object captured in the predicted real image to a ground truth probability corresponding to the object captured in the simulated image.
9 . The method of claim 8 , wherein the ground truth probability is:
determined by the simulator when the simulated image is generated.
10 . The method of claim 8 , wherein generating the perception loss based on comparing the predicted output for the predicted real image that is generated using the real perception model to the ground truth output for the simulated image that is generated using the simulator in generating the simulated image comprises:
determining a difference between the predicted probability and the ground truth probability; and
generating the perception loss as a function of the difference between the predicted probability and the ground truth probability.
11 . The method of claim 1 , wherein the predicted output for the predicted real image that is generated using the real perception model includes: a predicted bounding area corresponding to an object captured in the predicted real image, and a predicted probability corresponding to an object classification of the object captured in the predicted real image.
12 . The method of claim 11 , wherein generating the perception loss based on comparing the predicted output for the predicted real image that is generated using the real perception model to the ground truth output for the simulated image that is generated using the simulator in generating the simulated image comprises:
comparing the predicted bounding area corresponding to the object captured in the predicted real image to a ground truth bounding area corresponding to the object captured in the simulated image, and
comparing the predicted probability corresponding to the object classification of the object captured in the predicted real image to a ground truth probability corresponding to the object captured in the simulated image.
13 . The method of claim 12 , wherein generating the perception loss based on comparing the predicted output for the predicted real image that is generated using the real perception model to the ground truth output for the simulated image that is generated using the simulator in generating the simulated image further comprises:
generating the perception loss as a function of: comparing the predicted bounding area to the ground truth bounding area, and comparing the predicted probability to the ground truth probability.
14 . The method of claim 13 , wherein the ground truth bounding area and the ground truth probability are:
determined by the simulator when the simulated image is generated.
15 . The method of claim 1 , further comprising:
processing the predicted real image, using a simulation-to-real discriminator model of the generative adversarial network model, to determine whether the predicted real image is an actual real image or the predicted real image generated by the simulation-to-real generator model;
generating an adversarial loss based on determining whether the predicted real image is the actual real image or the predicted real image; and
wherein updating the simulation-to-real generator model is further based on the generated adversarial loss.
16 . The method of claim 1 , further comprising:
processing the predicted real image, using a real-to-simulation generator model of the generative adversarial network model or of an additional generative adversarial network model, to generate a cycled simulated image;
processing the cycled simulated image, using the real perception model or a simulated perception model, to generate further predicted output for the cycled simulated image; and
wherein generating the perception loss is further a function of comparisons between the ground truth output for the simulated image, the predicted output for the predicted real image, and the further predicted output for the cycled simulated image.
17 . The method of claim 16 , further comprising:
generating a cycle consistency loss based on comparing the simulated image that is generated using the simulator and the cycled simulated image that is generated using the real-to-simulation generator model of the generative adversarial network model or of the additional generative adversarial network model; and
wherein updating the simulation-to-real generator model is further based on the generated cycle consistency loss.
18 . The method of claim 1 , further comprising:
processing a real image, using a real-to-simulation generator model of the generative adversarial network model or of an additional generative adversarial network model, to generate a predicted simulated image that corresponds to the real image, wherein the real image is captured using one or more vision components;
processing the predicted simulated image, using a simulated perception model, to generate further predicted output for the predicted simulated image,
wherein the simulated perception model is trained based on simulated training instances, each of the simulated training instances comprising: simulated training instance input that includes a simulated image, and simulated training instance output that includes corresponding ground truth output for the simulated image;
generating a further perception loss based on comparing the further predicted output that is generated using the simulated perception model to further ground truth output for the real image; and
wherein updating the simulation-to-real generator model is further based on the generated further perception loss.
19 . A method implemented by one or more processors, the method comprising:
processing a simulated image, using a simulation-to-real generator model of a generative adversarial network model, to generate a predicted real image that corresponds to the simulated image, wherein the simulated image is generated using a simulator;
processing the predicted real image, using a real-to-simulation generator model of the generative adversarial network model, to generate a cycled simulated image that corresponds to the simulated image;
processing the simulated image, using a real perception model, to generate first predicted output for the simulated image;
processing the predicted real image, using the real perception model or a simulated perception model, to generate second predicted output for the predicted real image that corresponds to the simulated image;
processing the cycled simulated image, using the real perception model or the simulated perception model, to generate third predicted output for the cycled simulated image that corresponds to the simulated image;
generating a perception loss as a function of comparisons of the first predicted output for the simulated image, the second predicted output for the predicted real image that corresponds to the simulated image, and the third predicted output for the cycled simulated image that corresponds to the simulated image;
updating, based on the generated perception loss, the simulation-to-real generator model that was used to generate the predicted real image and based on processing the simulated image, the predicted real image that corresponds to the simulated image, and
the cycled simulated image that corresponds to the simulated image; and
subsequent to updating the simulation-to-real generator model:
generating a robotic control policy for a real robot based on simulated training data generated using the simulation-to-real generator model; and
causing the real robot to be controlled based on the generated robotic control policy.
20 . A method implemented by one or more processors, the method comprising:
training a simulation-to-real generator model of a generative adversarial network model, wherein training the simulation-to-real generator model comprises:
iteratively updating the simulation-to-real generator model based on a perception loss that is generated using a real perception model, wherein a given iteration, of iteratively updating the simulation-to-real generator model based on the perception loss that is generated using the real perception model, comprises:
processing a given simulated image, using the simulation-to-real generator model, to generate a given predicted real image that corresponds to the given simulated image, wherein the given simulated image is generated using a simulator,
processing the given predicted real image, using the real perception model, to generate given predicted output for the given predicted real image,
generating the perception loss based on comparing the given predicted output that is generated using the real perception model to given ground truth output for the given simulated image that is generated using the simulator in generating the given simulated image, and
updating, based on the generated perception loss, the simulation-to-real generator model that was used to generate the given predicted real image and based on processing the given simulated image; and
subsequent to training the simulation-to-real generator model:
generating a robotic control policy for a real robot based on simulated training data generated using the simulation-to-real generator model; and
causing the real robot to be controlled based on the generated robotic control policy.