System and method for generating training image data for supervised machine learning, and non-transitory recording medium
A system for generating training image data for supervised machine learning for training a defect classifier to be applied to visual inspection. The system arranges a defect model arbitrarily selected from a storage storing defect models obtained by modeling shapes and optical characteristics of defects and an inspection target surface model arbitrarily selected from a storage storing inspection target surface models obtained by modeling shapes and optical characteristics of inspection target surfaces, in any space in which ray tracing is performed by ray tracing simulation software in which an illumination optical system, an image pickup optical system, and an imaging sensor including a plurality of pixels are modeled. The system further traces a plurality of light rays, calculates illuminance at each of the pixels based on an intensity and a number of light rays entering the pixels, and generates pseudo image data as training image data based on the illuminance.
1 . A system for generating training image data for supervised machine learning for training defect classifier to be applied to visual inspection, the system comprising:
a defect model storage to store defect models obtained by modeling shapes and optical characteristics of defects;
an inspection target surface model storage to store inspection target surface models obtained by modeling shapes and optical characteristics of inspection target surfaces; and
a hardware processor to:
arrange a defect model arbitrarily selected from the defect models stored on the defect model storage and an inspection target surface model arbitrarily selected from the inspection target surface models stored on the inspection target surface model storage, in any space in which ray tracing is performed by ray tracing simulation software in which an illumination optical system, an image pickup optical system, and an imaging sensor including a plurality of pixels are modeled;
trace a plurality of light rays, and calculate illuminance at each of the pixels based on an intensity and a number of light rays entering the pixels; and
generate pseudo image data as the training image data based on the calculated illuminance,
wherein the illumination optical system includes strip-shaped lighting disposed so as to surround a vehicle body in a width direction,
the processor is further configured to generate the pseudo image data while at least one of an illumination optical system model or a model obtained by synthesizing the defect model and the inspection target surface model is moved by a minute amount and arranged at a plurality of different positions, and
wherein the inspection target surfaces are coated surfaces in which a clear layer is formed on a base coat layer, and an optical characteristic of the inspection target surface model includes a scattering characteristic of the base coat layer and a reflection characteristic of the clear layer.
2 . The system for generating training image data for supervised machine learning according to claim 1 , wherein each of the defect models stored on the defect model storage is given a label indicating a defect type of each defect model, and the pseudo image data generated by the hardware processor is given the same label as the label given to the defect model used in the generation of the pseudo image data.
3 . The system for generating training image data for supervised machine learning according to claim 1 , wherein the hardware processor arranges the defect model and the inspection target surface model at a plurality of different positions on the ray tracing simulation software and generates the pseudo image data.
4 . The system for generating training image data for supervised machine learning according to claim 1 , wherein the hardware processor has a function of either or both changing lightness and adding noise by performing image processing on the pseudo image data generated by the optical simulation.
5 . The system for generating training image data for supervised machine learning according to claim 1 , wherein the hardware processor creates a combined shape of the defect model and the inspection target surface model and places the combined shape of the defect model and the inspection target surface model in any space in which ray tracing is performed by the ray tracing simulation software.
6 . The system for generating training image data for supervised machine learning according to claim 1 , wherein the defect models are models of a coating defect in which a foreign matter is mixed into the clear layer.
7 . A method for generating training image data for supervised machine learning for training a defect classifier to be applied to visual inspection, the method comprising:
arbitrarily selecting a defect model from defect models obtained by modeling shapes and optical characteristics of defects and stored on a defect model storage;
arbitrarily selecting an inspection target surface model from inspection target surface models obtained by modeling shapes and optical characteristics of inspection target surfaces and stored on an inspection target surface model storage; and
arranging the selected defect model and the selected inspection target surface model in any space in which ray tracing is performed by ray tracing simulation software in which an illumination optical system, an image pickup optical system, and an imaging sensor including a plurality of pixels are modeled, tracing a plurality of light rays, calculating illuminance at each of the pixels based on an intensity and a number of light rays entering the pixels, and generating pseudo image data as the training image data based on the calculated illuminance,
generating the pseudo image data while at least one of an illumination optical system model or a model obtained by synthesizing the defect model and the inspection target surface model is moved by a minute amount and arranged at a plurality of different positions,
wherein the illumination optical system includes strip-shaped lighting disposed so as to surround a vehicle body in a width direction, and
wherein the inspection target surfaces are coated surfaces in which a clear layer is formed on a base coat layer, and an optical characteristic of the inspection target surface model includes a scattering characteristic of the base coat layer and a reflection characteristic of the clear layer.
8 . The method for generating training image data for supervised machine learning according to claim 7 , wherein each of the defect models stored on the defect model storage is given a label indicating a defect type of each defect model, and the generated pseudo image data is given the same label as the label given to the defect model used in the generation of the pseudo image data.
9 . The method for generating training image data for supervised machine learning according to claim 7 , wherein the defect model and the inspection target surface model are arranged at a plurality of different positions on the ray tracing simulation software, and the pseudo image data is generated.
10 . The method for generating training image data for supervised machine learning according to claim 7 , wherein either or both changing lightness and adding noise are achieved by performing image processing on the pseudo image data generated by the optical simulation.
11 . The method for generating training image data for supervised machine learning according to claim 7 , wherein a combined shape of the defect model and the inspection target surface model is created, and the combined shape of the defect model and the inspection target surface model is placed in any space in which ray tracing is performed by the ray tracing simulation software.
12 . The method for generating training image data for supervised machine learning according to claim 7 , wherein the defect models are models of a coating defect in which a foreign matter is mixed into the clear layer.
13 . A non-transitory computer-readable recording medium storing a program for causing a computer to execute generating training image data for supervised machine learning for training a defect classifier to be applied to visual inspection, the program causing the computer to execute:
arbitrarily selecting a defect model from defect models obtained by modeling shapes and optical characteristics of defects and stored on a defect model storage;
arbitrarily selecting an inspection target surface model from inspection target surface models obtained by modeling shapes and optical characteristics of inspection target surfaces and stored on an inspection target surface model storage;
arranging the selected defect model and the selected inspection target surface model in any space in which ray tracing is performed by ray tracing simulation software in which an illumination optical system, an image pickup optical system, and an imaging sensor including a plurality of pixels are modeled, tracing a plurality of light rays, calculating illuminance at each of the pixels based on an intensity and a number of light rays entering the pixels, and generating pseudo image data as the training image data based on the calculated illuminance, and
generating the pseudo image data while at least one of an illumination optical system model or a model obtained by synthesizing the defect model and the inspection target surface model is moved by a minute amount and arranged at a plurality of different positions,
wherein the illumination optical system includes strip-shaped lighting disposed so as to surround a vehicle body in a width direction, and
wherein the inspection target surfaces are coated surfaces in which a clear layer is formed on a base coat layer, and an optical characteristic of the inspection target surface model includes a scattering characteristic of the base coat layer and a reflection characteristic of the clear layer.
14 . The non-transitory computer-readable recording medium according to claim 13 , wherein each of the defect models stored on the defect model storage is given a label indicating a defect type of each defect model, and the program causes the computer to execute giving the generated pseudo image data the same label as the label given to the defect model used in the generation of the pseudo image data.
15 . The non-transitory computer-readable recording medium according to claim 13 , wherein the program causes the computer to execute arranging the defect model and the inspection target surface model at a plurality of different positions on the ray tracing simulation software and generating the pseudo image data.
16 . The non-transitory computer-readable recording medium according to claim 13 , wherein the program causes the computer to execute either or both changing lightness and adding noise by performing image processing on the pseudo image data generated by the optical simulation.
17 . The non-transitory computer-readable recording medium according to claim 13 , wherein the program causes the computer to execute creating a combined shape of the defect model and the inspection target surface model and placing the combined shape of the defect model and the inspection target surface model in any space in which ray tracing is performed by the ray tracing simulation software.
18 . The non-transitory computer-readable recording medium according to claim 13 , wherein the defect models are models of a coating defect in which a foreign matter is mixed into the clear layer.