Method and apparatus for generating vehicle damage image on the basis of GAN network
Embodiments of the present specification disclose a system and method for generating a vehicle damage image on the basis of a GAN model. During operation, the system obtains a real vehicle image, generates an intermediate image based on the real vehicle image by labeling a target box on the real vehicle image and removing a portion of the real vehicle image within the target box, and generates the vehicle damage image based on the intermediate image by inputting the intermediate image into a machine-learning model, which outputs the vehicle damage image by filling a local image indicating vehicle damage into the target box of the intermediate image.
1. A computer-executed method for generating a vehicle damage image, comprising:
obtaining a real vehicle image;
generating, by a computer, an intermediate image based on the real vehicle image by labeling a target box on the real vehicle image and removing a portion of the real vehicle image within the target box; and
generating the vehicle damage image based on the intermediate image by inputting the intermediate image into a machine-learning model, which outputs the vehicle damage image by filling a local image indicating vehicle damage into the target box of the intermediate image,
wherein the real vehicle image includes one or more local images indicating vehicle damage, and wherein labeling the target box comprises randomly selecting a labeling location corresponding to one of the one or more local images indicating vehicle damage.
2. The method of claim 1 , wherein labeling the target box comprises:
determining, based on statistics, a plurality of locations at which vehicle damage occurs with a high probability; and
randomly selecting, from the plurality of locations, a location for labeling the target box.
3. The method of claim 1 , wherein removing a portion of the real vehicle image comprises applying a mask, which comprises performing a dot-product operation on the real vehicle image and the mask.
4. The method of claim 1 , wherein the machine-learning model is a generative model in a Generative Adversarial Network (GAN) model, wherein the GAN model further comprises a discriminative model for determining whether an output image of the generative model is a real image and whether a local image indicating vehicle damage is in the target box of the output image, and wherein the generative model is trained based on the discriminative model, a second real vehicle image, and a second intermediate image generated based on the second real vehicle image.
5. The method of claim 4 , further comprising training the discriminative model by:
obtaining a plurality of positive samples and a plurality of negative samples, wherein a respective positive sample is a real image comprising a labeled target box, wherein the target box of the positive sample comprises a local image indicating vehicle damage, wherein the plurality of negative samples comprises a first negative sample being a non-real image comprising a labeled target box, and wherein the negative sample is obtained by replacing the local image within the target box of a real image with another local image; and
using the plurality of positive samples and the plurality of negative samples to train a classification model to be used as the discriminative model.
6. The method of claim 5 , wherein the first negative sample comprises at least one of the following features: component inconsistency within and outside the target box, vehicle model inconsistency within and outside the target box, color discontinuity within and outside the target box, and texture discontinuity within and outside the target box.
7. The method of claim 5 , wherein the plurality of negative samples further comprises a second negative sample being a real image that does not contain a local image indicating vehicle damage in its target box.
8. The method of claim 5 , wherein the discriminative model further comprises a semantic recognition model configured to determine whether the target box of a sample contains a local image indicating vehicle damage.
9. The method of claim 1 , further comprising: using the generated vehicle damage image to train a vehicle damage identification model for identifying damage to a vehicle based on a vehicle damage image.
10. A computer-executed apparatus for generating a vehicle damage image, comprising:
an image-acquisition unit configured to acquire a real vehicle image;
an intermediate-image generation unit configured to generate an intermediate image based on the real vehicle image by labeling a target box on the real vehicle image and removing a portion of the real vehicle image within the target box; and
a vehicle-damage-image generation unit configured to generate the vehicle damage image based on the intermediate image by inputting the intermediate image into a machine-learning model, which outputs the vehicle damage image by filling a local image indicating vehicle damage into the target box of the intermediate image,
wherein the machine-learning model is a generative model in a Generative Adversarial Network (GAN) model, wherein the GAN model further comprises a discriminative model for determining whether an output image of the generative model is a real image and whether a local image indicating vehicle damage is in the target box of the output image, and wherein the generative model is trained based on the discriminative model, a second real vehicle image, and a second intermediate image generated based on the second real vehicle image.
11. The apparatus of claim 10 , wherein the real vehicle image comprises one or more local images indicating vehicle damage, and wherein, while labeling the target box, the intermediate-image generation unit is configured to randomly select a labeling location corresponding to one of the one or more local images indicating vehicle damage.
12. The apparatus of claim 10 , wherein, while labeling the target box, the intermediate-image generation unit is configured to:
determine, based on statistics, a plurality of locations at which vehicle damage occurs with a high probability; and
randomly select, from the plurality of locations, a location for labeling the target box.
13. The apparatus of claim 10 , wherein, while removing a portion of the real vehicle image, the intermediate-image generation unit is configured to apply a mask on the real vehicle image, which comprises performing a dot-product operation on the real vehicle image and the mask.
14. The apparatus of claim 10 , further comprising a model training unit configured to:
obtain a plurality of positive samples and a plurality of negative samples, wherein a respective positive sample is a real image comprising a labeled target box, wherein the target box of the positive sample comprises a local image indicating vehicle damage, wherein the plurality of negative samples comprises a first negative sample being a non-real image comprising a labeled target box, and wherein the first negative sample is obtained by replacing the local image within the target box of a real image with another local image; and
use the plurality of positive samples and the plurality of negative samples to train a classification model to be used as the discriminative model.
15. The apparatus of claim 14 , wherein the first negative sample comprises at least one of the following features: component inconsistency within and outside the target box, vehicle model inconsistency within and outside the target box, color discontinuity within and outside the target box, and texture discontinuity within and outside the target box.
16. The apparatus of claim 14 , wherein the plurality of negative samples further comprise a second negative sample being a real image that does not contain a local image indicating vehicle damage in its target box.
17. The apparatus of claim 14 , wherein the discriminative model further comprises a semantic recognition model configured to determine whether the target box of a sample contains a local image indicating vehicle damage.
18. The apparatus of claim 10 , further comprising a second model training unit configured to use the generated vehicle damage image to train a vehicle damage identification model for identifying damage to a vehicle based on a vehicle damage image.
19. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for generating a vehicle damage image, the method comprising:
obtaining a real vehicle image;
generating an intermediate image based on the real vehicle image by labeling a target box on the real vehicle image and removing a portion of the real vehicle image within the target box; and
generating the vehicle damage image based on the intermediate image by inputting the intermediate image into a machine-learning model, which outputs the vehicle damage image by filling a local image indicating vehicle damage into the target box of the intermediate image,
wherein labeling the target box includes:
determining, based on statistics, a plurality of locations at which vehicle damage occurs with a high probability; and
randomly selecting, from the plurality of locations, a location for labeling the target box.
20. The non-transitory computer-readable storage medium according to claim 19 , wherein the real vehicle image comprises one or more local images indicating vehicle damage, and wherein labeling the target box includes randomly selecting a labeling location corresponding to one of the one or more local images indicating vehicle damage.
21. A computer-executed method for generating a vehicle damage image, comprising:
obtaining a real vehicle image;
generating, by a computer, an intermediate image based on the real vehicle image by labeling a target box on the real vehicle image and removing a portion of the real vehicle image within the target box; and
generating the vehicle damage image based on the intermediate image by inputting the intermediate image into a machine-learning model, which outputs the vehicle damage image by filling a local image indicating vehicle damage into the target box of the intermediate image,
wherein removing a portion of the real vehicle image includes applying a mask, which includes performing a dot-product operation on the real vehicle image and the mask.