Method for detecting non-problem domain data in a machine learning model
A method is provided for detecting non-problem domain (NPD) data in a machine learning (ML) model. The method includes training the ML model using problem domain (PD) training data. A second fully connected layer is added to the trained ML model in parallel with a first fully connected layer in the trained ML model. The trained ML model is retrained with NPD training data while preventing weights in the ML model from changing except for weights of the second fully connected layer. An inference operation is performed with the retrained ML model. Output vectors are received from the first and second fully connected layers via a Softmax layer. A metric is computed using the output vectors. The metric is compared to a threshold metric to determine if input samples are PD or NPD. An indication is provided when NPD data is detected. In another embodiment, a ML model is provided.
1 . A method for detecting non-problem domain input samples provided to a machine learning (ML), the method comprising:
training the ML model using problem domain training data to provide a trained ML model, wherein the trained ML model having a first fully connected layer and a first softmax layer for providing first classification predictions;
adding a second fully connected layer to the trained ML model in parallel with the first fully connected layer, and a second softmax layer connected to an output of the first fully connected layer for providing a first output vector and to an output of the second fully connected layer for providing a second output vector;
retraining the trained ML model using non-problem domain training data that is not in the problem domain to provide a retrained ML model for providing second classification predictions from the second softmax layer, wherein weights of all layers of the ML model are prevented from being changed during the retraining except for weights of the second fully connected layer;
providing a plurality of input samples to an input of the retrained ML model for an inference operation;
receiving an output vector from the second softmax layer of the retrained ML model;
computing a metric using the first and second output vectors from the second softmax layer; and
comparing the metric to a threshold metric to determine if the plurality of input samples are problem domain or non-problem domain.
2 . The method of claim 1 , wherein computing the metric further comprises taking a first mean value of the first output vector, taking a second mean value of the second output vector, and computing a proportion using the first and second mean values.
3 . The method of claim 2 , further comprising taking a logarithm of the proportion.
4 . The method of claim 1 , wherein computing the metric further comprises subtracting a first mean of the second output vector from a second mean of the first output vector.
5 . The method of claim 1 , wherein computing the metric further comprises subtracting a sum of the second output vector from a sum of the first output vector.
6 . The method of claim 1 , wherein computing the metric further comprises taking a proportion using a sum of the second output vector and a sum of the first output vector.
7 . The method of claim 6 , further comprising taking a logarithm of the proportion.
8 . The method of claim 1 , wherein the ML model comprises a neural network.
9 . The method of claim 1 , wherein the non-problem domain training data comprises classes of samples not related to classes of samples the problem domain training data.
10 . The method of claim 1 , further comprising taking an action to protect the ML model comprising one or more of limiting a number of input samples within a time window, reporting the copying attempt to a host processor, and substituting a lower quality ML model.
11 . A non-transitory storage medium comprising processor-readable instructions that, when executed, cause at least one processor core to provide a machine learning (ML) model, the ML model comprising:
an input layer for receiving input samples;
a plurality of intermediate layers coupled to the input layer;
a first fully connected layer coupled to an output of a final intermediate layer, wherein the first fully connected layer includes weights trained using problem domain data;
a second fully connected layer coupled to the output of the final intermediate layer, wherein the second fully connected layer includes weights trained using non-problem domain data;
a softmax layer coupled to an output of the first fully connected layer for output a first vector, and to an output of the second fully connected layers for outputting a second vector;
a non-problem domain detector coupled to receive the first and second vectors and to compute a metric using the first and second vectors; and
a comparator for comparing the metric to a threshold, wherein non-problem domain samples are detected when the metric does not compare favorably to the threshold.
12 . The non-transitory storage medium of claim 11 , wherein the metric further comprises taking a first mean value of elements of the first vector, taking a second mean value of elements of the second vector, and taking a proportion using the first and second mean values.
13 . The non-transitory storage medium of claim 12 , further comprising the non-problem domain detector taking a logarithm of the proportion.
14 . The non-transitory storage medium of claim 11 , wherein the metric further comprises a subtraction of a first mean value of elements of the second vector from a second mean value of elements of the first vector.
15 . The non-transitory storage medium of claim 11 , wherein the metric further comprises a subtraction of a sum of elements of the second output vector from a sum of elements of the first output vector.
16 . The non-transitory storage medium of claim 11 , wherein the metric further comprises taking a proportion using a sum of elements of the second vector and a sum of elements of the first vector.
17 . The non-transitory storage medium of claim 16 , further comprising taking a logarithm of the proportion.
18 . The non-transitory storage medium of claim 11 , wherein the ML model is used for image classification.
19 . The non-transitory storage medium of claim 11 , further comprising an action taken to protect the ML model comprises one or more of limiting the number of samples that can be input in a given time window, reporting the copying attempt to a host processor, and substituting a lower quality ML model.
20 . The non-transitory storage medium of claim 11 , wherein the ML model comprises a neural network.