AI-assisted human data augmentation and continuous training for machine learning models
A method is provided for training at least one classifier model used by an artificial intelligence (AI) system to recognize each of a set of objects and to assign each of the set of objects to a class. The method includes training the at least one classifier model on a training dataset, thereby producing at least one trained classifier model; using the at least one trained classifier model to detect and classify each member of a set of objects, thereby generating a set of inferences, wherein each inference includes (a) a cropped image of a classified object, (b) the classified object's inferred class, and (c) a confidence score associated with the inferred classification; examining the set of inferences with a machine implemented audit trigger, wherein the audit trigger identifies a subset of the set of inferences whose members have (i) a confidence score that falls below a predetermined threshold value, or (ii) a missing classification; and if the identified subset has at least one member, subjecting the identified subset to a human audit, thereby yielding a corrected set of observations, wherein, for each member of the corrected set of observations, the inferred class of the corresponding member of the set of inferences is replaced with a corrected class. The corrected set of observations is then added to a training dataset and used to improve the future accuracy of the classifier model.
1 . A method for training at least one classifier model used by an artificial intelligence (AI) system to recognize each of a set of objects and to assign each of the set of objects to a class, the method comprising:
training the at least one classifier model on a training dataset, thereby producing at least one trained classifier model;
using the at least one trained classifier model to detect and classify each member of a set of objects, thereby generating a set of inferences, wherein each inference includes
(a) a cropped image of a classified object,
(b) the classified object's inferred classification, and
(c) a confidence score associated with the inferred classification;
examining the set of inferences with a machine implemented audit trigger, wherein the audit trigger identifies a subset of the set of inferences whose members have (i) a confidence score that falls below a predetermined threshold value, or (ii) a missing inferred classification;
if the identified subset has at least one member, subjecting the identified subset to a human audit, thereby yielding a corrected set of observations, wherein, for each member of the corrected set of observations, the inferred classification of the corresponding member of the set of inferences is replaced with a corrected classification;
examining the corrected set of observations with a benefit scoring process, thereby identifying at least one proposed modification to the training dataset; and
incorporating the at least one proposed modification into the training dataset in a subsequent iteration of the method.
2 . The method of claim 1 , wherein each member of the set of objects is a consumer packaged good (CPG).
3 . The method of claim 1 , wherein the benefit scoring process excludes from the at least one proposed modification observations associated with an image that is blurry or occluded.
4 . The method of claim 1 , wherein the benefit scoring process implements a deep learning model that learns over time which observations to include in the training dataset utilized in a subsequent iteration of the method.
5 . The method of claim 1 , wherein the benefit scoring process implements an algorithmic process that examines characteristics selected from the group consisting of image quality and class accuracy.
6 . The method of claim 1 , wherein the benefit scoring process implements a human augmentation model that includes human participation in the decision.
7 . The method of claim 1 , wherein the benefit scoring process selects corrected observations for inclusion in the training dataset utilized in a subsequent iteration of the method using at least one selection criteria selected from the group consisting of (a) image quality, and (b) the number of training images already in the training dataset which correspond to the product identified in the corrected observation.
8 . The method of claim 1 , wherein the audit trigger identifies all members of the set of inferences for which the confidence score falls below a predetermined threshold value.
9 . The method of claim 1 , wherein the audit trigger identifies all members of the set of inferences which are missing a classification.
10 . The method of claim 1 , wherein the audit trigger identifies all members of the set of inferences for which the classifier model cannot identify the class to which the object belongs.
11 . The method of claim 1 , wherein the audit trigger identifies all members of the set of inferences for which the difference between the confidence scores of the top two inferences for the same object is below a predetermined threshold value.
12 . The method of claim 1 , wherein the audit trigger identifies all members of the set of inferences whose members have a confidence score that falls below a predetermined threshold value.
13 . The method of claim 1 , wherein the subset of the set of inferences is one of a set of commonly confused classes due to fine-grained differences in packaging.
14 . The method of claim 1 , further comprising:
publishing a set of observations to a group of data consumers, wherein the published set of observations includes (a) members of the set of inferences exclusive of the subset of the set of inferences, and (b) the corrected set of observations.
15 . The method of claim 1 , wherein the at least one classifier model is a set of cooperating models which identify and qualify distinct aspects of images to be classified.
16 . The method of claim 15 , wherein the at least one classifier model is a sequence of models that perform the steps of:
detecting and cropping object images within larger frames or videos;
identifying similarities of the detected object images to existing classes; and
differentiating between the detected object images and other similar classes of objects.
17 . The method of claim 16 , wherein examining the corrected set of observations with a benefit scoring process includes examining the corrected set of observations with a plurality of benefit scoring processes.
18 . The method of claim 17 , wherein incorporating the at least one proposed modification into the training dataset in a subsequent iteration of the method results in the generation of a plurality of training datasets, and wherein each of the plurality of training datasets is used to train the classifier model based on a distinct set of criteria.
19 . The method of claim 1 , wherein the audit trigger is an algorithmic process.
20 . The method of claim 1 , wherein the audit trigger is a rules-based process.
21 . The method of claim 1 , wherein the audit trigger implements a deep-learning approach to adjust its selection criteria based on results achieved in prior iterations of the method.
22 . The method of claim 21 wherein, if at least one of said prior iterations of the method produces a subset containing at least k members for which the inference was correct, and if k>m>0, wherein k and m are integers and m is a predetermined threshold value, then the audit trigger increases the predetermined threshold value for confidence scores in at least one subsequent iteration of the method.
23 . The method of claim 1 , further comprising:
using a proportion of corrected inferences resulting from the human audit process to compute recognition accuracy metrics.
24 . The method of claim 23 , further comprising:
using the computed recognition accuracy metrics to monitor the performance of the at least one classifier model.
25 . The method of claim 23 , further comprising:
using the computed recognition accuracy metrics to train at least one component of the AI system.
26 . The method of claim 25 , wherein the computed accuracy metrics include metrics selected from the group consisting of product categories, geographic areas, and overall summary metrics.
27 . A tangible, non-transient medium containing suitable programming instructions which, when processed by at least one computer processor, perform the method of claim 1 .
28 . A method for training at least one classifier model used by an artificial intelligence (AI) system to recognize each of a set of objects and to assign each of the set of objects to a class, the method comprising:
training the at least one classifier model on a training dataset, thereby producing at least one trained classifier model;
using the at least one trained classifier model to detect and classify each member of a set of objects, thereby generating a set of inferences, wherein each inference includes
(a) a cropped image of a classified object,
(b) the classified object's inferred classification, and
(c) a confidence score associated with the inferred classification;
examining the set of inferences with a machine implemented audit trigger, wherein the audit trigger identifies a subset of the set of inferences whose members have (i) a confidence score that falls below a predetermined threshold value, or (ii) a missing inferred classification; and
if the identified subset has at least one member, subjecting the identified subset to a human audit, thereby yielding a corrected set of observations, wherein, for each member of the corrected set of observations, the inferred classification of the corresponding member of the set of inferences is replaced with a corrected classification;
wherein the audit trigger identifies all members of the set of inferences for which the difference between the confidence scores of the top two inferences for the same object is below a predetermined threshold value.