Active learning management system for automated inspection systems
An example method for selecting product images for training a machine-learning model includes obtaining product images to include in an image population; receiving an indication of an image selection strategy for determining if a product image is to be included in a set of images of interest; determining image transforms based on configuration data for the indicated image selection strategy, wherein the image transforms perform image manipulation operations to obtain transformed image data for each of the product images in the image population; selecting a subset of images from the image population for inclusion in the set of images of interest based on the indicated image selection strategy and the transformed image data; determining one or more descriptive labels and applying the one or more descriptive labels to the respective sets of images; and training an inspection model for a product inspection system based on the labeled images.
1 . A method for selecting product images for training a machine-learning model to inspect images of a product, the method comprising:
obtaining, by an image gatherer of an active learning framework from an image repository, product images to include in an image population;
receiving, by an image selector, an indication of an image selection strategy from a plurality of image selection strategies, each of the image selection strategies defining selection operations for determining if a product image is to be included in a set of images of interest;
instantiating, by the image selector, a selection strategy task process corresponding to the image selection strategy, the selection strategy task process configured to perform the selection operations of the image selection strategy;
determining, by the selection strategy task process, one or more image transforms from a plurality of image transforms based on configuration data for the indicated image selection strategy;
in response to determining that one or more of the product images in the image population have not been previously transformed according to the one or more image transforms, instantiating one or more image transform task processes corresponding to the one or more image transforms, wherein the one or more image transform task processes are configured to perform image manipulation operations to obtain the transformed image data for each of the one or more of the product images in the image population;
selecting, by the selection strategy task process, a subset of images from the image population for inclusion in the set of images of interest based on the selection operations for the indicated image selection strategy and the transformed image data;
determining, by an image labeler, an indication of one or more descriptive labels and applying the one or more descriptive labels to the respective sets of images; and
training an inspection model for a product inspection system based on the set of images of interest and corresponding labels of the set of images of interest.
2 . The method of claim 1 , further comprising:
in response to determining that one or more of the plurality of product images in the image population have been previously transformed according to an image transform of the one or more image transforms, bypassing execution of the image transformation task process associated with the image transform for the one or more of the plurality of product images and retrieving the transformed image data from a storage location.
3 . The method of claim 1 , wherein the image selection strategy comprises a first image selection strategy, the subset of images comprises a first subset of images, and the selection strategy task process comprises a first selection strategy task process, wherein the method further comprises:
receiving, by an image selector, an indication of a second image selection strategy from the plurality of image selection strategies, the second image selection strategy comprising second selection operations for determining if the product image is to be included in the set of images of interest;
instantiating, by the image selector, a second selection strategy task process corresponding to the second image selection strategy; and
selecting, by the second selection strategy task process, a second subset of images from the image population based on the second selection operations for the second image selection strategy and the transformed image data;
wherein the set of images of interest includes the first subset of images and the second subset of images.
4 . The method of claim 3 , wherein the inspection model comprises a first inspection model, and wherein the method further comprises:
determining a first performance metric for the first inspection model;
determining a second performance metric for a second inspection model;
comparing the first performance metric and the second performance metric; and
providing an indication of whether to deploy the second inspection model to an inspection system for a production line based on the comparison.
5 . The method of claim 1 , further comprising:
adding a second selection strategy to the plurality of image selection strategies, wherein a configuration for the second selection strategy includes a set of fixed properties, a set of user editable properties, and a list of image transformations associated with the second image selection strategy.
6 . The method of claim 1 , wherein determining the indication of the one or more descriptive labels to be applied to respective sets of images of the images of interest comprises one of:
receiving the indication of the one or more descriptive labels from a user via a user interface; and
automatically determining the one or more descriptive labels.
7 . The method of claim 1 , wherein obtaining, from the image repository, product images comprises obtaining the product images and zero or more associated metadata associated with the product images from an inspection system for a production line for the product.
8 . The method of claim 7 , wherein obtaining the product images from the inspection system comprises obtaining the product images based on scores assigned to the product images by an inspection model of the inspection system.
9 . The method of claim 1 , wherein selecting, by the selection strategy task process, the subset of images from the image population for inclusion in the set of images of interest comprises:
receiving a number of images N that are to be selected from the image population; and
generating one or more D-dimensional spheres (D-spheres) of radius R around each of one or more datapoints associated with labeled images, wherein R is determined according to a standard deviation of the data multiplied by D;
iteratively performing, until N images are selected for the set of images of interest, operations comprising:
generating an additional D-sphere of radius R around a datapoint associated with an unlabeled image;
including images associated with datapoints from outside of the generated D-spheres in a candidate set of images; and
selecting an image from the candidate set of images for inclusion in the set of images of interest.
10 . The method of claim 9 , further comprising:
in response to determining that a number of the candidate images is less than a threshold percentage of images in the image population, reducing the radius R.
11 . An active learning management system for selecting product images for training a machine-learning model to inspect images of a product, the active learning management system comprising:
a memory; and
processing circuitry configured to:
obtain, from an image repository, product images to include in an image population;
receive an indication of an image selection strategy from a plurality of image selection strategies, each of the image selection strategies defining selection operations for determining if a product image is to be included in a set of images of interest;
instantiate a selection strategy task process corresponding to the image selection strategy, the selection strategy task process configured to perform the selection operations of the image selection strategy;
determine, by the selection strategy task process, one or more image transforms from a plurality of image transforms based on configuration data for the indicated image selection strategy;
in response to a determination that one or more of the product images in the image population have not been previously transformed according to the one or more image transforms, instantiate one or more image transform task processes corresponding to the one or more image transforms, wherein the one or more image transform task processes are configured to perform image manipulation operations to obtain the transformed image data for each of the one or more of the product images in the image population;
select, by the selection strategy task process, a subset of images from the image population for inclusion in the set of images of interest based on the selection operations for the indicated image selection strategy and the transformed image data;
determine an indication of one or more descriptive labels and apply the one or more descriptive labels to the respective sets of images; and
train an inspection model for a product inspection system based on the set of images of interest and corresponding labels of the set of images of interest.
12 . The active learning management system of claim 11 , wherein the processing circuitry is further configured to:
in response to a determination that one or more of the plurality of product images in the image population have been previously transformed according to an image transform of the one or more image transforms, bypass execution of the image transformation task process associated with the image transform for the one or more of the plurality of product images and retrieve the transformed image data from a storage location.
13 . The active learning management system of claim 11 , wherein the image selection strategy comprises a first image selection strategy, the subset of images comprises a first subset of images, and the selection strategy task process comprises a first selection strategy task process, wherein the processing circuitry is further configured to:
receive an indication of a second image selection strategy from the plurality of image selection strategies, the second image selection strategy comprising second selection operations for determining if the product image is to be included in the set of images of interest;
instantiate a second selection strategy task process corresponding to the second image selection strategy; and
select, by the second selection strategy task process, a second subset of images from the image population based on second selection operations and the transformed image data;
wherein the set of images of interest includes the first subset of images and the second subset of images.
14 . The active learning management system of claim 13 , wherein the inspection model comprises a first inspection model, and wherein the processing circuitry is further configured to:
determine a first performance metric for the first inspection model;
determine a second performance metric for a second inspection model;
compare the first performance metric and the second performance metric; and
provide an indication of whether to deploy the second inspection model to an inspection system for a production line based on the comparison.
15 . The active learning management system of claim 11 , wherein the processing circuitry is further configured to:
add a second selection strategy to the plurality of image selection strategies, wherein a configuration for the second selection strategy includes a set of fixed properties, a set of user editable properties, and a list of image transformations associated with the second image selection strategy.
16 . The active learning management system of claim 11 , wherein to determine the indication of the one or more descriptive labels to be applied to respective sets of images of the images of interest comprises one of:
receive the indication of the one or more descriptive labels from a user via a user interface; and
automatically determine the one or more descriptive labels.
17 . The active learning management system of claim 11 , wherein to obtain, from the image repository, product images comprises to obtain the product images and zero or more associated metadata associated with the product images from an inspection system for a production line for the product.
18 . The active learning management system of claim 11 , wherein to select, by the selection strategy task process, the subset of images from the image population for inclusion in the set of images of interest comprises:
receive a number of images N that are to be selected from the image population; and
generate one or more D-dimensional spheres (D-spheres) of radius R around each of one or more datapoints associated with labeled images, wherein R is determined according to a standard deviation of the data multiplied by D;
iteratively perform, until N images are selected for the set of images of interest, operations comprising:
generate an additional D-sphere of radius R around a datapoint associated with an unlabeled image;
include images associated with datapoints from outside of the generated D-spheres in a candidate set of images; and
select an image from the candidate set of images for inclusion in the set of images of interest.
19 . The active learning management system of claim 18 , wherein the processing circuitry is further configured to:
in response to a determination that a number of the candidate images is less than a threshold percentage of images in the image population, reduce the radius R.