Multi-party machine learning using a database cleanroom
Embodiments of the present disclosure may provide a data sharing system implemented as a local application in a consumer database of a distributed database. The local application can include a training function and a scoring function to train a machine learning model on provider and consumer data, and generate output data by applying the trained machine learning model on input data. The input data can include data portions from a consumer database and a provider database that are joined to create a joined dataset for scoring.
1 . A method performed by executing instructions on at least one hardware processor, the method comprising:
accessing, by a consumer database account, a provider-account shared-instance database provided by a provider database account, the provider-account shared-instance database comprising provider-account training data; and
initiating training, by the consumer database account, of a machine-learning-model instance of a machine learning model, the initiating of the training comprising:
creating the machine-learning-model instance of the machine learning model;
identifying a set union of the provider-account training data stored within the provider-account shared-instance database and consumer-account training data stored in the consumer database account, the identifying of the set union of the provider-account training data and the consumer-account training data stored in the consumer database account comprising executing a join function on the provider-account training data and the consumer-account training data to generate a combined training dataset; and
training the machine-learning-model instance subsequent to the creation of the machine-learning-model instance with at least the combined training dataset.
2 . The method of claim 1 , further comprising installing, by the consumer database account, the provider-account shared-instance database that comprises a shared instance of a provider account database in the consumer database account.
3 . The method of claim 2 , wherein the provider-account shared-instance database resides in the provider database account.
4 . The method of claim 2 , wherein the shared instance comprises a first schema, wherein data stored within the first schema is hidden from the consumer database account and the provider database account, and wherein a training function stored within the first schema is hidden from the consumer database account and the provider database account.
5 . The method of claim 4 , wherein the shared instance comprises a second schema, wherein the second schema comprises the machine-learning-model instance created by the training function, and wherein data stored within the second schema is hidden from the consumer database account and the provider database account.
6 . The method of claim 1 , wherein the provider-account shared-instance database comprises and a training function, the training being initiated by invoking the training function.
7 . The method of claim 1 , wherein the machine-learning-model instance is not accessible to be used by the consumer database account.
8 . The method of claim 1 , wherein the machine-learning-model instance is not accessible by the consumer database account.
9 . The method of claim 1 , wherein the machine-learning-model instance is trained with the identified set union of the provider-account training data and the consumer-account training data.
10 . The method of claim 9 , wherein the provider-account training data accessed locally by the consumer database account via the provider-account shared-instance database.
11 . The method of claim 6 , wherein an underlying functionality of the training function is not accessible to the consumer database account.
12 . The method of claim 4 , wherein:
the first schema further comprises provider-account scoring data and a scoring function;
an underlying functionality of the scoring function is not accessible to the consumer database account; and
the method further comprises:
generating consumer-account scoring data by inputting, into the machine-learning-model instance, consumer-account input data that is stored in the consumer database account; and
storing the consumer-account scoring data in the consumer database account.
13 . The method of claim 1 , wherein the machine-learning-model instance is also not accessible to the provider database account.
14 . The method of claim 1 , further comprising the provider database account revoking access to the machine-learning-model instance from the consumer database account.
15 . The method of claim 1 , wherein both the provider and the consumer database accounts reside in a distributed database.
16 . The method of claim 1 , wherein:
the consumer database account resides in a first networked database platform; and
the provider database account resides in a second networked database platform.
17 . The method of claim 16 , wherein the first networked database platform and the second networked database platform are in different geographic regions.
18 . The method of claim 5 , wherein the training of the machine-learning-model instance comprises:
invoking the training function, the invoking of the training function automatically triggering operations of creating the second schema, identifying the set union, and training of the machine-learning-model instance.
19 . A computer system comprising:
at least one hardware processor; and
one or more non-transitory computer readable storage media containing instructions that, when executed by the at least one hardware processor, cause the computer system to perform operations comprising:
accessing, by a consumer database account, a provider-account shared-instance database provided by a provider database account, the provider-account shared-instance database comprising provider-account training data; and
initiating training, by the consumer database account, of a machine-learning-model instance of a machine learning model, the initiating of the training comprising:
creating the machine-learning-model instance of the machine learning model;
identifying a set union of the provider-account training data stored within the provider-account shared-instance database and consumer-account training data stored in the consumer database account, the identifying of the set union of the provider-account training data and the consumer-account training data stored in the consumer database account comprising executing a join function on the provider-account training data and the consumer-account training data to generate a combined training dataset; and
training the machine-learning-model instance subsequent to the creation of the machine-learning-model instance with at least the combined training dataset.
20 . One or more non-transitory computer readable storage media containing instructions that, when executed by at least one hardware processor of a computer system, cause the computer system to perform operations comprising:
accessing, by a consumer database account, a provider-account shared-instance database provided by a provider database account, the provider-account shared-instance database comprising provider-account training data; and
initiating training, by the consumer database account, of a machine-learning-model instance of a machine learning model, the initiating of the training comprising:
creating the machine-learning-model instance of the machine learning model;
identifying a set union of the provider-account training data stored within the provider-account shared-instance database and consumer-account training data stored in the consumer database account, the identifying of the set union of the provider-account training data and the consumer-account training data stored in the consumer database account comprising executing a join function on the provider-account training data and the consumer-account training data to generate a combined training dataset; and
training the machine-learning-model instance subsequent to the creation of the machine-learning-model instance with at least the combined training dataset.