Integrating machine learning classification models and machine learning anomaly models
In example embodiments, a hybrid classification/anomaly machine learning architecture is provided that combines a classification model and an anomaly model to perform an engineering task. The classification model and the anomaly model may be used in parallel, and their inference results compared, with their consistency used to improve confidence, and their inconsistency used to detect when additional training or other improvement is required and to capture data useful in such additional training/improvement.
1 . A method for using a hybrid classification/anomaly machine learning architecture to inspect infrastructure for damage, comprising:
accessing input data that includes representations of infrastructure, wherein the infrastructure includes at least one of bridges, buildings, roads, railways, electrical and communications networks, or equipment;
submitting the input data in parallel to a classification model and an anomaly model of a classification/anomaly machine learning architecture executing on one or more computing devices;
producing, by the classification model, a first inference result that indicates whether a class of damage is detected in the representations of infrastructure;
producing, by the anomaly model, a second inference result that indicates whether something other than undamaged condition is detected in the representations of infrastructure;
comparing, by an application executing on the one or more computing devices, the first inference result and the second inference result for consistency, wherein the first inference result and the second inference result are considered to have consistency when the first inference result indicates a class of damage is detected and the second inference result indicates something other than undamaged condition is detected or when the first inference result indicates no class of damage is detected and the second inference result indicates undamaged condition is detected; and
in response to consistency between the first inference result and the second inference result and the first and second inference results being consistent positive results,
reporting the class of damage to the infrastructure in a user interface of the application.
2 . The method of claim 1 , further comprising:
in response to consistency between the first inference result and the second inference result,
adding the input data to a feature dataset used to train the classification model and/or adding the input data to an anomaly dataset used to train the anomaly model.
3 . The method of claim 1 , further comprising:
in response to inconsistency between the first inference result and the second inference result and the first inference result indicating a class of damage is detected and the second inference result indicating undamaged condition is detected is detected,
adding the input data to a feature buffer, and obtaining a subject matter expert (SME) assessment of the input data that indicates whether there is a class of damage to inject ground truth.
4 . The method of claim 3 , further comprising:
in response to the SME assessment indicating no class of damage,
adding the input data to a feature dataset used to train the classification model and/or adding the input data to an anomaly dataset used to train the anomaly model.
5 . The method of claim 3 , further comprising:
in response to if the SME assessment indicating a class of damage,
activating an anomaly dataset purification process that is configured to remove contaminated samples from the anomaly dataset used to train the anomaly model.
6 . The method of claim 1 , further comprising:
in response to inconsistency between the first inference result and the second inference result and if the first inference result indicating no class of damage is detected and the second inference result indicating something other than undamaged condition is detected,
adding the input data to an anomaly buffer, and obtaining a subject matter expert (SME) assessment of the input data that indicates whether there is something other than undamaged condition to inject ground truth.
7 . The method of claim 6 , further comprising:
in response to the SME assessment indicating undamaged condition,
adding the input data to a feature dataset used to train the classification model and/or adding the input data to an anomaly dataset used to train the anomaly model.
8 . The method of claim 6 , further comprising:
in response to the SME assessment indicating something other than undamaged condition,
obtaining a label for the input data from the SME, and adding the input data to a feature dataset used to train the classification model.
9 . The method of claim 1 , further comprising
executing a classification model improvement process that is configured to add normal samples from the anomaly dataset used to train the anomaly model to the feature dataset used to train the classification model.
10 . The method of claim 1 , further comprising
executing an anomaly model improvement process that is configured to add normal samples from the feature dataset used to train the classification model to the anomaly dataset used to train the anomaly model.
11 . The method of claim 1 , further comprising:
activating a feature dataset purification process that is configured to remove contaminated samples from the feature dataset used to train the classification model.
12 . A system that uses a hybrid classification/anomaly machine learning architecture to inspect infrastructure for damage, comprising:
one or more computing devices including one or more processors;
an input dataset;
a classification model;
an anomaly model; and
an application that when executed on the one or more processors of the one or more computing devices is operable to:
access input data that includes representations of infrastructure, wherein the infrastructure includes at least one of bridges, buildings, roads, railways, electrical and communications networks, or equipment;
submit the input data from the input dataset in parallel to the classification model and the anomaly model,
compare a first inference result from the classification model that indicates whether a class of damage is detected in the input data and a second inference result from the anomaly model that indicates whether something other than undamaged condition is detected in the input data for consistency, wherein the first inference result and the second inference result are considered to have consistency when the first inference result indicates a class of damage is detected and the second inference result indicates something other than undamaged condition is detected or when the first inference result indicates no class of damage is detected and the second inference result indicates undamaged condition is detected, and
in response to consistency between the first inference result and the second inference result, at least one of a) report a detected feature or b) add the input data to a feature dataset used to train the classification model and/or add the input data to an anomaly dataset used to train the anomaly model.
13 . The system of claim 12 , wherein the application when executed on one or more processors of one or more computing devices is further operable to:
in response to inconsistency between the first inference result and the second inference result, add the input data to a buffer, obtain a subject matter expert (SME) assessment of the input data that indicates whether there is a feature or an anomaly to inject ground truth, and use the input data to train the classification model and/or to train the anomaly model.