FINGERPRINT SECURITY USING AN AUXILIARY MACHINE-LEARNING TOOL
Methods and apparatus are disclosed for providing security for a target machine-learning (ML) tool and its host system. Input data is fed in parallel to a second ML tool. Fingerprints of the second ML tool are used to monitor changes in the second tool. Fingerprint changes above a threshold indicate anomalous input data and warn of possible threat to the target tool. Anomaly detection enables diagnosis and remediation. Compact fingerprints are easy to handle, and hide details of the underlying tool. Concurrently, fingerprints are large enough to be sensitive to localized variations within the tool. Alternative embodiments monitor fingerprints of the target tool itself. Further embodiments monitor input or output data streams for sensitive data using a trained ML classifier. Variations and applications are disclosed.
1 . A computer-implemented method, comprising:
(a) inputting data directed toward a first machine-learning (ML) tool to a second ML tool;
(b) computing a loss function on output of the second ML tool;
(c) updating parameters of the second ML tool based on results of the computing;
(d) determining a current fingerprint of the second ML tool with the updated parameters;
(e) comparing the current fingerprint with a prior fingerprint of the second ML tool; and
in a first case having a distance measure between the current and prior fingerprints above a predetermined threshold:
(f) outputting a notification indicating anomalous data.
2 . The computer-implemented method of claim 1 , further comprising:
performing additional iterations of acts (a)-(e).
3 . The computer-implemented method of claim 1 , wherein the first ML tool or the second ML tool is part of a microservice in a network of microservices configured as a copilot.
4 . The computer-implemented method of claim 1 , wherein the first ML tool is configured to undergo training based on the data of act (a).
5 . The computer-implemented method of claim 1 , wherein the first ML tool is configured to perform inference based on the data of act (a).
6 . The computer-implemented method of claim 1 , wherein the determining the current fingerprint comprises:
for each of multiple clusters of the parameters of the second ML tool:
determining a composite weight data structure; and
combining the composite weight data structures of the multiple clusters into the current fingerprint.
7 . The computer-implemented method of claim 1 , wherein the prior fingerprint comprises:
a common fingerprint used as the prior fingerprint over multiple iterations of acts (a)-(e); or
a fingerprint determined at act (d) on an immediately preceding iteration.
8 . The computer-implemented method of claim 1 , wherein the current and prior fingerprints specify respective vectors or points in a multiple dimensional space and the distance measure comprises:
a measure of Cartesian distance between the respective vectors or points.
9 . The computer-implemented method of claim 1 , further comprising, in the first case, performing one or more diagnostic actions comprising:
at least one action to identify one or more portions of the second ML tool which were most impacted by the anomalous data.
10 . The computer-implemented method of claim 1 , further comprising, in the first case, performing one or more diagnostic actions comprising:
at least one action to identify one or more portions of data inputted to the second ML tool, between the prior fingerprint and the current fingerprint, which contributed to the distance measure exceeding the predetermined threshold.
11 . The computer-implemented method of claim 1 , further comprising, in the first case, performing one or more diagnostic actions comprising:
at least one action to assess an impact of the anomalous data on the first ML tool.
12 . The computer-implemented method of claim 1 , further comprising, prior to act (a):
(v) training the first machine-learning (ML) tool according to a first regimen;
(w) training the second ML tool according to a second regimen associated with the first regimen;
(x) determining an output tolerance for a given tool among the first and second ML tool;
(y) obtaining a procedure to compute a fingerprint of the second ML tool; and
(z) determining a threshold fingerprint variation corresponding to the output tolerance for the given tool.
13 . One or more computer-readable media storing instructions which, when executed on one or more hardware processors, cause the one or more hardware processors to perform operations comprising:
(a) inputting data directed toward a first machine-learning (ML) tool to a second ML tool;
(b) computing a loss function on output of the second ML tool;
(c) updating parameters of the second ML tool based on results of the computing;
(d) determining a current fingerprint of the second ML tool with the updated parameters;
(e) comparing the current fingerprint with a prior fingerprint of the second ML tool; and
in a first case having a distance measure between the current and prior fingerprints above a predetermined threshold:
(f) outputting a notification indicating anomalous data.
14 . The one or more computer-readable media of claim 13 , wherein operation (d) further comprises:
for each of multiple clusters of the parameters of the second ML tool:
determining a composite weight data structure; and
combining the composite weight data structures of the multiple clusters into the current fingerprint, wherein the combining comprises:
applying weight factors to respective ones of the composite weight data structures.
15 . The one or more computer-readable media of claim 13 , wherein the second ML tool is a neural network ML tool, the parameters of the second ML tool are allocated among a plurality of clusters, each of the clusters belonging to exactly one among a first group of the clusters and a second group of the clusters, each of the parameters reflecting a weight of a corresponding edge joining two cells of the neural network ML tool, and operation (d) further comprises:
calculating respective first composite weight data structures with high fidelity for each of the first group of clusters;
calculating respective second composite weight data structures with low fidelity for each of the second group of clusters; and
combining the first and second composite weight data structures into the current fingerprint.
16 . The one or more computer-readable media of claim 13 , wherein the second ML tool is a neural network ML tool, each of the parameters of the second ML tool reflecting a weight of a corresponding first-tier edge joining two cells of the neural network ML tool, some of the parameters are grouped into a plurality of first-tier clusters, and others of the parameters reflect weights of corresponding first-tier edges joining two cells of distinct first-tier clusters, and operation (d) further comprises:
calculating respective first composite weight data structures for each of the first-tier clusters;
constructing a second-tier graph comprising a plurality of second-tier vertices, each second-tier vertex corresponding to a respective one of the first-tier clusters, and a plurality of second-tier edges, each second-tier edge joining a respective pair of the second-tier vertices;
for each second-tier edge, joining respective first and second second-tier vertices corresponding to first and second first-tier clusters:
assigning a weight to the each second-tier edge, based on a number and/or weights of the first-tier edges joining the first and second first-tier clusters;
constructing one or more second composite weight data structures for the second-tier graph; and
combining the first and second composite weight data structures into the current fingerprint.
17 . A system, comprising:
one or more hardware processors with memory coupled thereto; and
computer-readable media storing instructions which, when executed by the one or more hardware processors, cause the one or more hardware processors to perform operations comprising:
(a) inputting data directed toward a first machine-learning (ML) tool to a second ML tool;
(b) computing a loss function on output of the second ML tool;
(c) updating parameters of the second ML tool based on results of the computing;
(d) determining a current fingerprint of the second ML tool with the updated parameters;
(e) comparing the current fingerprint with a prior fingerprint of the second ML tool; and
in a first case having a distance measure between the current and prior fingerprints above a predetermined threshold:
(f) outputting a notification indicating anomalous data.
18 . The system of claim 17 , wherein the operations further comprise, in the first case, at least one remediation action to control one or more sources of data inputted to the second ML tool between the prior fingerprint and the current fingerprint.
19 . The system of claim 17 , wherein the operations further comprise, in the first case, at least one remediation action to restore at least part of the first ML tool or the second ML tool to an earlier snapshot.
20 . The system of claim 17 , wherein the operations further comprise, in the first case, at least one remediation action to warn recipients of output from the first ML tool between the prior fingerprint and the current fingerprint.