Appearance inspection apparatus and appearance inspection method
It is possible to quickly present an alternative model without causing deterioration in usability of a user. Learning data is input to a machine learning network to train the machine learning network, and a first inference model configured to perform quality determination of input images is generated. The input images sequentially input to the first inference model and quality determination results of the input images are stored. A process of inputting the plurality of stored input images to the machine learning network to train the machine learning network and generating a second inference model is executed in the background of quality determination processing at the time of inspection. A display screen configured to display quality determination performance of the second inference model is displayed on a display section.
1 . An appearance inspection apparatus that inputs a workpiece image obtained by capturing an image of a workpiece, which is an object to be inspected, to a machine learning network and determines quality of the workpiece based on the input workpiece image, the appearance inspection apparatus comprising:
a learning section that inputs learning data to the machine learning network to train the machine learning network and generates a first inference model configured to perform quality determination of input images relating to a specific type of the workpiece;
an inspection section that sequentially inputs the input images to the first inference model generated by the learning section and performs the quality determination of the input images;
a storage section that stores the input images sequentially input to the first inference model generated by the learning section and quality determination results of the input images, the quality determination results being based on an output from the first inference model; and
a display controller that causes a display section to display the quality determination results obtained by the inspection section,
a setting section configured to set a trigger condition and a select condition, wherein
the trigger condition is for automatically starting a background learning which is executed in a background of performing the quality determination and is specified by at least one of date and time, a change in a characteristic amount of the input images input to the first inference model, and a statistical change in the quality determination results of the input images,
the select condition is for automatically selecting a learning image used for the background learning, the learning image being selected among the input images input to the first inference model and stored in the storage, the select condition including at least one of a capturing time period of the learning image and an attribute of the learning image,
wherein the learning section
selects the learning image among the plurality of input images stored in the storage section in accordance with the select condition, the learning image satisfying the select condition,
starts the background learning to generate a second inference model in accordance with the trigger condition executing a process of inputting the learning image to the machine learning network to train the machine learning network, and
the display controller causes the display section to display a user interface indicating quality determination performance of the second inference model, the quality determination performance of the second inference model relating to the specific type of the workpiece, and
the user interface includes
a non-defective product image which is determined based on the output from the second inference model, and
a defective product image which is determined based on the output from the second inference model.
2 . The appearance inspection apparatus according to claim 1 , wherein the display controller causes the display section to display a display screen configured to compare quality determination performance of the first inference model and the quality determination performance of the second inference model.
3 . The appearance inspection apparatus according to claim 1 , wherein
the learning section generates a plurality of the second inference models corresponding to different select conditions set by the setting section, by inputting images selected in accordance with the select conditions to the machine learning network to perform background learning, and
the display controller causes the display section to display a display screen configured to compare the quality determination performance among the plurality of second inference models.
4 . The appearance inspection apparatus according to claim 1 , wherein
the setting section is capable of setting a predetermined period as the select condition, and
the learning section selects an input image captured within the predetermined period as the learning image.
5 . The appearance inspection apparatus according to claim 1 , wherein
the setting section is capable of setting a first select condition and a second select condition as the select condition,
the first select condition relates to a capturing time period in which an input image selected as the learning image being used to generate the second inference model has been captured,
the second select condition relates to an attribute of an input image used to generate the second inference model, and
the learning image satisfies both the second select condition and the first select condition.
6 . The appearance inspection apparatus according to claim 5 , wherein
the second select condition is being a non-defective product image corresponding to a non-defective product.
7 . The appearance inspection apparatus according to claim 5 , wherein
the second select condition is being a defective product image corresponding to a defective product.
8 . The appearance inspection apparatus according to claim 5 , wherein the second select condition is a characteristic amount of an image being a predetermined value or more.
9 . The appearance inspection apparatus according to claim 1 , wherein
the setting section is capable of setting a third condition related to an amount of input images used for the background learning to generate the second inference model, and
the learning section inputs the learning image to the machine learning network during the background learning in accordance with the third condition.
10 . The appearance inspection apparatus according to claim 9 , wherein
the third condition is a ratio of a learning image amount to an inspection image amount,
the inspection image amount is an amount of the plurality of input images input to the first inference model and stored in the storage section,
the learning image amount is the amount of the input images used for background learning to generate the second inference model, and
the learning section inputs the learning images to the machine learning network during the background learning, so that the amount of learning images corresponds to the ratio.
11 . The appearance inspection apparatus according to claim 1 , wherein the learning section also uses the learning data, which has been used in generating the first inference model, when generating the second inference model.
12 . The appearance inspection apparatus according to claim 1 , wherein
the learning section inputs verification image data, to which quality information has been added in advance, to each of the first inference model and the second inference model and executes quality determination of the verification image data, and calculates a first match rate, which is a match rate between the quality information of the verification image data and a quality determination result obtained by the first inference model, and a second match rate which is a match rate between the quality information of the verification image data and a quality determination result obtained by the second inference model, and
the display controller provides display regions of the first match rate and the second match rate in a display screen configured to compare quality determination performance of the first inference model and the quality determination performance of the second inference model, and causes the display section to display the display screen.
13 . An appearance inspection method of inputting a workpiece image obtained by capturing an image of a workpiece, which is an object to be inspected, to a machine learning network and determining quality of the workpiece based on the input workpiece image, the appearance inspection method comprising:
a first learning step of inputting learning data to the machine learning network to train the machine learning network and generating a first inference model configured to perform quality determination of input images;
an inspection step of sequentially inputting the input images to the first inference model generated in the first learning step and performing the quality determination of the input images relating to a specific type of the workpiece;
a storage step of storing the input images sequentially input to the first inference model generated in the first learning step and quality determination results of the input images, the quality determination results being based on an output from the first inference model;
a setting step of setting a trigger condition and a select condition, the trigger condition being for automatically starting a background learning which is executed in a background of performing the quality determination and is specified by at least one of date and time, a change in a characteristic amount of the input images input to the first inference model, and a statistical change in the quality determination results of the input images, the select condition being for automatically selecting a learning image used for the background learning, the learning image being selected among the input images input to the first inference model and stored in the storage, the select condition including at least one of a capturing time period of the learning image and an attribute of the learning image;
a selection step of selecting the learning image among the plurality of input images stored by the storage step, in accordance with the select condition, the learning image satisfying the select condition;
a background learning step to generate a second inference model, which is started in accordance with the trigger condition, including executing a process of inputting a plurality of the learning images to the machine learning network to train the machine learning network; and
a display step of causing a display section to display a user interface indicating quality determination performance of the second inference model, the quality determination performance of the second inference model relating to the specific type of the workpiece, the user interface including a non-defective product image which is determined based on the output from the second inference model, and a defective product image which is determined based on the output from the second inference model.