Third-party analytic and machine learning model validation
Systems and methods for validating analytics or machine learning models using secure multi-party computation (SMPC) are disclosed. The system operates with a data owner party providing a validation data set and an analytic owner party supplying a specification of the analytic or machine learning model. The analytic owner party converts the specification into an oblivious computation and compiles it into a circuit of operations compatible with an SMPC protocol. The data owner party downloads the compiled specification and constructs a validation circuit to execute the model against the validation data set without revealing the data or model details. The results of the SMPC are evaluated to produce performance statistics of the model, ensuring privacy for both parties' data throughout the process.
1 . A method for validating analytics or machine learning (ML) models using secure multi-party computation (SMPC), the method comprising:
a data owner party providing a validation data set;
an analytic owner party receiving a specification of an analytic or ML model, the specification including parameters marked as sensitive, the analytic owner party converting the specification into an oblivious computation, the analytic owner party compiling the oblivious computation into a circuit of operations compatible with an SMPC protocol;
the data owner party connecting to the analytic owner party to download the compiled specification, the data owner party constructing a validation circuit, the validation circuit containing a computation necessary to run the ML model or analytic against each item in the validation data set; and
the data owner party and the analytic owner party executing an SMPC to evaluate the validation circuit, at the end of which a designated party receives output statistics indicating a performance of the analytic or ML model against the validation data set.
2 . The method of claim 1 , wherein the specification includes a detailed representation of a structure and operations of the analytic or ML model, comprising a neural network architecture with specific layer configurations, activation functions, and weight matrices.
3 . The method of claim 1 , wherein the specification includes hyperparameters and optimization parameters, comprising learning rates, regularization coefficients, and batch sizes, that govern training and performance of the ML model.
4 . The method of claim 1 , wherein the SMPC protocol is a secure computation protocol that enables execution of computations over encrypted data without revealing inputs to any party involved.
5 . The method of claim 1 , wherein the oblivious computation replaces conditional logic with oblivious logic.
6 . The method of claim 5 , wherein the oblivious logic computes a Boolean value for each possible branch of the conditional logic, indicating whether that branch should be followed for a current input; a final value for each possible branch of the conditional logic; and a final output of the conditional logic, wherein the final output is computed by either summing products of each Boolean value and the corresponding final value for each branch, or performing an XOR operation on a result of AND-ing each Boolean value with the corresponding final value for each branch.
7 . The method of claim 1 , wherein the validation circuit marks all data in the validation data set as private inputs from the data owner party, all parameter inputs to the analytic or ML model as private inputs from the analytic owner party, and all outputs as private outputs to the data owner party.
8 . The method of claim 1 , wherein the analytic owner party receives the output statistics.
9 . A system for performing validation of analytics or machine learning (ML) models using secure multi-party computation (SMPC), the system comprising:
a data owner component of a processor that is configured by a data owner party, that provides a validation data set;
an analytic owner component of the processor that is configured by an analytic owner party, that receives a specification of an analytic or ML model, the specification including parameters marked as sensitive, and converts the specification into an oblivious computation which is compiled into a circuit of operations compatible with an SMPC protocol;
a connection module of the processor allowing the data owner component to connect to the analytic owner component and download the compiled specification, then construct a validation circuit; and
an execution module of the processor which allows both the data owner component and the analytic owner component to execute an SMPC to evaluate the validation circuit, with final output statistics indicating a performance of the analytic or ML model against the validation data set being received by a designated party.
10 . The system of claim 9 , wherein the specification includes a detailed representation of a structure and operations of the analytic or ML model, comprising a neural network architecture with specific layer configurations, activation functions, and weight matrices.
11 . The system of claim 9 , wherein the specification includes hyperparameters and optimization parameters, comprising learning rates, regularization coefficients, and batch sizes, that govern training and performance of the ML model.
12 . The system of claim 9 , wherein the SMPC protocol is a secure computation protocol that enables execution of computations over encrypted data without revealing inputs to any party involved.
13 . The system of claim 9 , wherein the oblivious computation replaces any conditional logic with oblivious logic.
14 . The system of claim 13 , wherein the oblivious logic computes a Boolean value for each possible branch of the conditional logic, indicating whether that branch should be followed for a current input; a final value for each possible branch of the conditional logic; and a final output of the conditional logic, wherein the final output is computed by either summing products of each Boolean value and the corresponding final value for each branch, or performing an XOR operation on a result of AND-ing each Boolean value with the corresponding final value for each branch.
15 . The system of claim 9 , wherein the validation circuit marks all data in the validation data set as private inputs from the data owner party, all parameter inputs to the analytic or ML model as private inputs from the analytic owner party, and all outputs as private outputs to the data owner party, wherein the analytic owner party receives the final output statistics.
16 . The system of claim 9 , further comprising the data owner party providing a training data set, the data owner party and the analytic owner party executing the SMPC to evaluate a training circuit, the training circuit constructed similarly as the validation circuit, to adapt the parameters of the ML model based on the training data set without revealing sensitive information of either party.
17 . The system of claim 9 , wherein the validation circuit further comprises a full computation necessary to:
run the ML model or analytic against each of a plurality of items in the validation data, to obtain results;
compare the results to desired results, to obtain final results; and
compute desired overall statistics for the final results.
18 . The system of claim 9 , wherein the validation circuit marks all data in the validation data set as private inputs from the data owner party, and the validation circuit marks all parameter inputs to the analytic or ML model as private inputs from the analytic owner party.
19 . A method for validating analytics or machine learning (ML) models using secure multi-party computation (SMPC), performed by a computer system, the method comprising:
receiving a validation data set;
receiving a specification of an analytic or ML model, including parameters marked as sensitive;
converting the specification into an oblivious computation;
compiling the oblivious computation into a circuit of operations compatible with an SMPC protocol;
constructing a validation circuit containing a computation necessary to run the analytic or ML model against each item in the validation data set; and
executing an SMPC to evaluate the validation circuit.
20 . The method of claim 19 , further comprising receiving output statistics indicating a performance of the analytic or ML model against the validation data set.