Method for image analysis
A method and system for synthetic data generation and analysis includes generating a synthetic dataset. A set of parameters is determined and scenarios are generated from the parameters that represent three-dimensional scenes. Synthetic images are rendered for the scenarios. A synthetic dataset may be formed to have a controlled variation in attributes of synthetic images over a synthetic dataset. The synthetic dataset may be used for training or evaluating a machine learning model.
1. A computer-implemented method comprising:
determining respective parameter values of a plurality of scenario parameters;
generating a plurality of scenarios based on the respective parameter values of the plurality of scenario parameters, wherein the plurality of scenarios represents three-dimensional scenes defined by the plurality of scenario parameters, wherein one scenario of the plurality of scenarios includes simulated sensor data that simulates sensor data received from one or more radar sensors;
rendering a plurality of synthetic images that respectively correspond to the plurality of scenarios;
augmenting the plurality of synthetic images to generate a plurality of augmented synthetic images; and
generating a synthetic dataset using the plurality of synthetic images and the plurality of augmented synthetic images.
2. The method of claim 1 , wherein augmenting the plurality of synthetic images comprises:
defining the plurality of synthetic images as a set;
augmenting a first subset of the set; and
leaving un-augmented a second subset of the set; and
wherein generating the synthetic dataset comprises combining the first subset and the second subset as the synthetic dataset.
3. The method of claim 2 , wherein defining the first subset and the second subset reduces a size of a dataset that is needed to achieve a desired level of model performance by excluding some of the plurality of synthetic images.
4. The method of claim 1 , wherein augmenting the plurality of synthetic images enhances realism of the synthetic images by implementing a modeled optical effect.
5. The method of claim 4 , wherein the modeled optical effect uses one or more point spread functions in an image domain.
6. The method of claim 1 , wherein augmenting the plurality of synthetic images comprises generating the plurality of augmented synthetic images by adding a variation to two or more of the plurality of synthetic images, each synthetic image of the plurality of synthetic images being varied by an augmentation parameter to create the plurality of augmented synthetic images.
7. The method of claim 6 , wherein the augmentation parameter is varied within a range from a maxima to a minima associated with a value of the augmentation parameter extracted from a real-world image dataset.
8. The method of claim 6 , wherein the augmentation parameter is sampled from a low discrepancy sequence to reduce unintended and/or undesired correlation across the synthetic dataset.
9. The method of claim 6 , wherein the augmentation parameter is a translation parameter that performs translation in an x or y direction of a subframe of an image frame.
10. The method of claim 6 , wherein the augmentation parameter is an image transformation parameter for brightness, contrast, dynamic range, or other parameters that varies a visible quality of the synthetic image.
11. The method of claim 6 , wherein the augmentation parameter is a geometric transformation parameter including governing translation, rotation, scaling, or flipping.
12. The method of claim 6 , wherein the augmentation parameter is an optical simulation parameter including governing lens properties, parametrized optical distortions related to virtual lenses or other media, lens flare, chromatic aberrations, other aberrations, or point spread function shape.
13. The method of claim 6 , wherein the augmentation parameter is a simulated sensor or camera parameter including governing exposure level, sensor dynamic range, sensor black level, light response curve, static noise, temporal noise, shot noise, photon noise, color filter array, color filter array arrangement, color filter array filter characteristics, or demosaicing.
14. A system comprising one or more processors and memory operably coupled with the one or more processors, wherein the memory stores instructions that, in response to execution of the instructions by the one or more processors, cause the one or more processors to perform operations including:
determining respective parameter values of a plurality of scenario parameters;
generating a plurality of scenarios based on the respective parameter values of the plurality of scenario parameters, wherein the plurality of scenarios represents three-dimensional scenes defined by the plurality of scenario parameters, wherein one scenario of the plurality of scenarios includes simulated sensor data that simulates sensor data received from one or more radar sensors;
rendering a plurality of synthetic images that respectively correspond to the plurality of scenarios;
augmenting the plurality of synthetic images to generate a plurality of augmented synthetic images; and
generating a synthetic dataset using the plurality of synthetic images and the plurality of augmented synthetic images.
15. The system of claim 14 , wherein the one or more processors to perform the operations including:
defining the plurality of synthetic images as a set;
augmenting a first subset of the set; and
leaving un-augmented a second subset of the set; and
wherein generating the synthetic dataset comprises combining the first subset and the second subset as the synthetic dataset.
16. The system of claim 15 , wherein defining the first subset and the second subset reduces a size of a dataset that is needed to achieve a desired level of model performance by excluding some of the plurality of synthetic images.
17. The system of claim 14 , wherein augmenting the plurality of synthetic images enhances realism of the synthetic images by implementing modeled optical effects.
18. The system of claim 17 , wherein the realism of the synthetic images is enhanced by implementing modeled optical effects using one or more point spread functions in an image domain.
19. The system of claim 14 , wherein augmenting the plurality of synthetic images comprises generating the plurality of augmented synthetic images by adding a variation to two or more of the plurality of synthetic images, each synthetic image of the plurality of synthetic images being varied by an augmentation parameter to create the plurality of augmented synthetic images.
20. The system of claim 19 , wherein the augmentation parameter is varied within a range from a maxima to a minima associated with a value of the augmentation parameter extracted from a real-world image dataset.