System and method for ensuring that the results of machine learning models can be audited
Systems, methods, and other embodiments associated with auditing the results of a machine learning model are described. In one embodiment, a method accesses original time series data and machine learning estimates of the original time series data. The method generates reconstituted time series data from the machine learning estimates by reversing operations of a machine learning model trained for generating the machine learning estimates from the original time series data. The method detects tampering (or corruption) in the original time series data based on a difference between the original time series data and reconstituted time series data. And, the method generates an electronic verification report that indicates whether the tampering (or corruption) is detected in the original time series data.
1. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by at least a processor of a computer, cause the computer to:
access original time series data and machine learning estimates of the original time series data;
generate reconstituted time series data from the machine learning estimates by reversing operations of a machine learning model trained for generating the machine learning estimates from the original time series data;
detect tampering in the original time series data based on a difference between the original time series data and reconstituted time series data; and
generate an electronic verification report that indicates whether the tampering is detected in the original time series data.
2. The non-transitory computer-readable media of claim 1 , wherein the instructions for reversing operations of the machine learning model further cause the computer to:
identify the operations of the machine learning model;
determine an inverse operation for each of the operations that undoes the operation;
configure the inverse operations to be performed in a reverse order from that of the machine learning model; and
execute the inverse operations in the reverse order on the machine learning estimates to produce the reconstituted time series data.
3. The non-transitory computer-readable media of claim 1 , wherein the instructions for detecting tampering in the original time series data further cause the computer to compare values from the reconstituted time series data and original time series data pairwise using a sequential probability ratio test that detects the tampering as an anomaly.
4. The non-transitory computer-readable media of claim 1 , wherein the instructions further cause the computer to, prior to generating reconstituted time series data:
select a training data set from the original time series data that was captured prior to an update to the original time series data; and
train the machine learning model to generate estimates consistent with correlation patterns learned between time series signals of the training data set.
5. The non-transitory computer-readable media of claim 1 , wherein the instructions further cause the computer to, prior to generating reconstituted time series data, reduce complexity of the trained machine learning model by limiting the trained machine learning model to major components of a matrix that represents the trained machine learning model.
6. The non-transitory computer-readable media of claim 1 , wherein the machine learning estimates are multivariate state estimates, and the machine learning model is a multivariate state estimation model.
7. The non-transitory computer-readable media of claim 1 , wherein the instructions further cause the computer to display the electronic verification report on a graphical user interface.
8. A computing system, comprising:
a processor;
a memory operably connected to the processor;
one or more non-transitory computer-readable media operably connected to the processor and memory and storing computer-executable instructions that when executed by at least the processor of the computing system cause the computing system to:
access original time series data and machine learning estimates of the original time series data;
generate reconstituted time series data from the machine learning estimates by reversing operations of a machine learning model trained for generating the machine learning estimates from the original time series data;
detect corruption in the original time series data based on a difference between the original time series data and reconstituted time series data; and
generate an electronic verification report that indicates whether the corruption is detected in the original time series data.
9. The computing system of claim 8 , wherein the instructions for reversing operations of the machine learning model further cause the computing system to:
identify the operations of the machine learning model;
determine an inverse operation for each of the operations that undoes the operation;
configure the inverse operations to be performed in a reverse order from that of the machine learning model; and
execute the inverse operations in the reverse order on the machine learning estimates to produce the reconstituted time series data.
10. The computing system of claim 8 , wherein the instructions for detecting corruption in the original time series data further cause the computing system to compare values from the reconstituted time series data and original time series data pairwise using a sequential probability ratio test that detects the corruption as an anomaly.
11. The computing system of claim 8 , wherein the instructions further cause the computing system to, prior to generating reconstituted time series data:
select a training data set from the original time series data that was captured prior to an update to the original time series data; and
train the machine learning model to generate estimates consistent with correlation patterns learned between time series signals of the training data set.
12. The computing system of claim 8 , wherein the instructions further cause the computing system to, prior to generating reconstituted time series data, reduce complexity of the trained machine learning model by limiting the trained machine learning model to major components of a matrix that represents the trained machine learning model.
13. The computing system of claim 8 , wherein the machine learning estimates are multivariate state estimates, and the machine learning model is a multivariate state estimation model.
14. The computing system of claim 8 , wherein the instructions further cause the computing system to display the electronic verification report on a graphical user interface.
15. A computer-implemented method, comprising:
accessing original time series data and machine learning estimates of the original time series data;
generating reconstituted time series data from the machine learning estimates by reversing operations of a machine learning model trained for generating the machine learning estimates from the original time series data;
detecting tampering in the original time series data based on a difference between the original time series data and reconstituted time series data; and
generating an electronic verification report that indicates whether the tampering is detected in the original time series data.
16. The computer-implemented method of claim 15 , wherein reversing operations of the machine learning model further comprises:
identifying the operations of the machine learning model;
determining an inverse operation for each of the operations that undoes the operation;
configuring the inverse operations to be performed in a reverse order from that of the machine learning model; and
executing the inverse operations in the reverse order on the machine learning estimates to produce the reconstituted time series data.
17. The computer-implemented method of claim 15 , wherein detecting tampering in the original time series data further comprises comparing values from the reconstituted time series data and original time series data pairwise using a sequential probability ratio test that detects the tampering as an anomaly.
18. The computer-implemented method of claim 15 , further comprising, prior to generating reconstituted time series data:
select a training data set from the original time series data that was captured prior to an update to the original time series data; and
training the machine learning model to generate estimates consistent with correlation patterns learned between time series signals of the training data set.
19. The computer-implemented method of claim 15 , further comprising, prior to generating reconstituted time series data, reducing complexity of the trained machine learning model by limiting the trained machine learning model to major components of a matrix that represents the trained machine learning model.
20. The computer-implemented method of claim 15 , wherein the machine learning estimates are multivariate state estimates, and the machine learning model is a multivariate state estimation model.