Generating synthetic training data for perception machine learning models using simulated environments

1. A method, comprising:

generating a set of candidate training data based on a simulated environment and a first set of environmental parameters for the simulated environment;

training a machine learning model based on the set of candidate training data;

obtaining a set of segmentations based on the machine learning model and a set of test data, wherein the set of segmentations indicate features of the simulated environment identified by the machine learning model;

determining whether a mean intersection-over-union (MIOU) of the set of segmentations has increased by more than a threshold amount;

in response to determining that the MIOU has increased by more than the threshold amount, generating a next set of candidate training data based on a next simulated environment and a next set of environmental parameters for the simulated environment; and

in response to determining that the MIOU has not increased by the threshold amount, determining whether to select a next set of environmental parameters; and

in response to determining that the next set of environmental parameters should be selected, generating the next set of candidate training data based on the next set of environmental parameters.

2. The method of claim 1 , further comprising:

in response to determining that the MIOU has not increased by the threshold amount, determining whether to change an increment for the set of environmental parameters; and

in response to determining that the increment for the set of environmental parameters should be changed, changing the increment and generating the next set of candidate training data based on the changed increment.

3. The method of claim 1 , wherein:

the simulated environment comprises a set of routes; and

the set of candidate training data comprises a set of images of obtained based on the set of routes within the simulated environment.

4. The method of claim 3 , wherein generating the set of candidate training data based on the simulated environment comprises:

running a simulation engine, wherein:

the set of images depict views of the simulated environment along the set of routes; and

the views of the simulated environment correspond to a location of a sensor within a vehicle.

5. The method of claim 1 , wherein:

the first set of environmental parameters indicate one or more of a set of locations, a set of shapes, a set of orientations, a set of colors, a set of textures and a set of sizes for a set of non-deterministically generated objects; and

generating the set of candidate training data based on the simulated environment comprises:

generating the simulated environment based on one or more of the set of locations, the set of shapes, the set of orientations, the set of colors, the set of textures and the set of sizes for the set of non-deterministically generated objects.

6. The method of claim 1 , wherein:

the first set of environmental parameters indicate a non-deterministically selected weather condition; and

generating the set of candidate training data based on the simulated environment comprises:

generating the simulated environment based on the non-deterministically selected weather condition.

7. The method of claim 1 , wherein:

the first set of environmental parameters indicate a non-deterministically selected lighting condition; and

generating the set of candidate training data based on the simulated environment comprises:

generating the simulated environment based on the non-deterministically selected lighting condition.

8. The method of claim 1 , wherein:

the first set of environmental parameters indicate a set of non-deterministically selected colors; and

generating the set of candidate training data based on the simulated environment comprises:

generating the simulated environment based on the set of non-deterministically selected colors.

9. The method of claim 1 , wherein the first set of environmental parameters indicate ranges of values for each environmental parameter in the first set of environmental parameters and increments for each environmental parameter in the first set of environmental parameters.

10. The method of claim 1 , further comprising:

selecting the first set of environmental parameters from a plurality of environmental parameters, based on previous MIOUs.

11. The method of claim 1 , further comprising:

controlling operation of a vehicle based on the machine learning model.

12. A apparatus, comprising:

a memory configured to store data; and

a processing device coupled to the memory, the processing device configured to:

generate a set of candidate training data based on a simulated environment and a first set of environmental parameters for the simulated environment;

train a machine learning model based on the set of candidate training data;

obtain a set of segmentations based on the machine learning model and a set of test data, wherein the set of segmentations indicate features of the simulated environment identified by the machine learning model;

determine whether a mean intersection-over-union (MIOU) of the set of segmentations has increased by more than a threshold amount; and

in response to determining that the MIOU has increased by more than the threshold amount, generate a next set of candidate training data based on a next simulated environment and a next set of environmental parameters for the simulated environment; and

in response to determining that the MIOU has not increased by the threshold amount, determining whether to change an increment for the set of environmental parameters; and

in response to determining that the increment for the set of environmental parameters should be changed, changing the increment and generating the next set of candidate training data based on the changed increment.

13. The apparatus of claim 12 , wherein the processing device is further configured to:

in response to determining that the MIOU has not increased by the threshold amount, determine whether to select a next set of environmental parameters; and

in response to determining that the next set of environmental parameters should be selected, generate the next set of candidate training data based on the next set of environmental parameters.

14. The apparatus of claim 12 , wherein:

the simulated environment comprises a set of routes; and

the set of candidate training data comprises a set of images of obtained based on the set of routes within the simulated environment.

15. The apparatus of claim 12 , wherein:

the first set of environmental parameters indicate one or more of a set of locations, a set of shapes, a set of orientations, a set of colors, a set of textures, and a set of sizes for a set of non-deterministically generated objects; and

to generate the set of candidate training data based on the simulated environment the processing device is further configured to:

generate the simulated environment based on one or more of the set of locations, the set of shapes, the set of orientations, the set of colors, the set of textures, and the set of sizes for the set of non-deterministically generated objects.

16. The apparatus of claim 12 , wherein:

the first set of environmental parameters indicate a non-deterministically selected weather condition; and

to generate the set of candidate training data based on the simulated environment the processing device is further configured to:

generate the simulated environment based on the non-deterministically selected weather condition.

17. The apparatus of claim 12 , wherein:

the first set of environmental parameters indicate a non-deterministically selected lighting condition; and

to generate the set of candidate training data based on the simulated environment the processing device is further configured to:

generate the simulated environment based on the non-deterministically selected lighting condition.

18. A non-transitory computer-readable storage medium including instructions that, when executed by a processing device, cause the processing device to perform operations comprising:

generating a set of candidate training data based on a simulated environment and a first set of environmental parameters for the simulated environment;

training a machine learning model based on the set of candidate training data;

obtaining a set of segmentations based on the machine learning model and a set of test data, wherein the set of segmentations indicate features of the simulated environment identified by the machine learning model;

determining whether a mean intersection-over-union (MIOU) of the set of segmentations has increased by more than a threshold amount; and

in response to determining that the MIOU has increased by more than the threshold amount, generating a next set of candidate training data based on a next simulated environment and a next set of environmental parameters for the simulated environment, wherein the first set of environmental parameters indicate ranges of values for each environmental parameter in the first set of environmental parameters and increments for each environmental parameter in the first set of environmental parameters.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the operations further comprise:

in response to determining that the MIOU has not increased by the threshold amount, determine whether to select a next set of environmental parameters; and

in response to determining that the next set of environmental parameters should be selected, generate the next set of candidate training data based on the next set of environmental parameters.

20. The non-transitory computer-readable storage medium of claim 18 , wherein the operations further comprise:

in response to determining that the MIOU has not increased by the threshold amount, determining whether to change an increment for the set of environmental parameters; and

in response to determining that the increment for the set of environmental parameters should be changed, changing the increment and generating the next set of candidate training data based on the changed increment.