IP Library Granted Patent US 11,681,688
Granted Patent B2
US 11,681,688 · App. 17/186,748 · Granted Jun 20, 2023

Immutable and decentralized storage of computer models

Inventors: Ilana Alexandra Golbin (Pasadena, CA); Joseph David Voyles (Louisville, KY); Kris Douglas Kersey (Pittsburgh, PA); Thomas Joseph Foth (Melrose, MA)
Assignee: PricewaterhouseCoopers LLP
G06F16/2379G06N20/00H04L67/104H04L67/1095H04L67/1097
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,681,688
App. No.
17/186,748
Granted
Jun 20, 2023
Kind
B2
Abstract

The present disclosure relates generally to storing computer models, and more specifically to a platform for achieving replicability of a computer model (e.g., a trained machine-learning algorithm) by storing and providing access to data associated with the computer model using an immutable and decentralized ledger system (e.g., a blockchain ledger) and a distributed database. An exemplary computer-enabled method for storing a computer model, the method comprises: receiving data associated with the computer model; generating one or more asset files based on the data associated with the computer model; generating one or more hash values corresponding to the one or more asset files; generating one or more of location trackers corresponding to the one or more asset files; generating a ledger entry comprising the one or more hash values and the one or more location trackers; and adding the ledger entry to a blockchain ledger.

Claims (108)

1. A computer-enabled method for storing a computer model, the method comprising:

receiving data associated with the computer model;

generating a plurality of asset files based on the data associated with the computer model, wherein generating the plurality of asset files comprises:

generating a first asset file comprising a source data set for the computer model, wherein the source data set comprises a training data set, a testing data set, and a validation data set;

generating a second asset file comprising experimentation data, wherein the experimentation data comprises model development code for the computer model; and

generating a third asset file comprising a binary record representing a model structure and learned functions of the computer model;

generating a plurality of hash values corresponding to the plurality of asset files;

generating a plurality of location trackers corresponding to the plurality of asset files;

generating a ledger entry comprising the plurality of hash values and the one or more plurality of location trackers; and

adding the ledger entry to a blockchain ledger.

2. The method of claim 1 , further comprising: distributing each of the one or more asset files based on a corresponding location tracker.

3. The method of claim 2 , wherein distributing each of the one or more asset file comprises: generating multiple copies of each of the one or more asset files, and transmitting the multiple copies to multiple nodes in a network for storage.

4. The method of claim 1 , wherein the ledger is a distributed ledger, wherein a first copy of the ledger is stored on a first node of a peer-to-peer network, and wherein a second copy of the ledger is stored on a second node of the peer-to-peer network.

5. The method of claim 1 , wherein the computer model is a trained machine learning model.

6. The method of claim 1 , wherein a location tracker of the one or more location trackers comprises a distribution scheme of a corresponding asset file.

7. The method of claim 6 , wherein the distribution scheme indicates locations at which a plurality of copies of the corresponding asset file are stored.

8. The method of claim 1 , wherein registering the ledger entry to the blockchain ledger comprises adding the ledger entry to the blockchain ledger.

9. An electronic device, comprising:

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

receiving data associated with a computer model;

generating a plurality of asset files based on the data associated with the computer model, wherein generating the plurality of asset files comprises:

generating a first asset file comprising a source data set for the computer model, wherein the source data set comprises a training data set, a testing data set, and a validation data set;

generating a second asset file comprising experimentation data, wherein the experimentation data comprises model development code for the computer model; and

generating a third asset file comprising a binary record representing a model structure and learned functions of the computer model;

generating a plurality of hash values corresponding to the plurality of asset files;

generating a plurality of location trackers corresponding to the plurality of asset files;

generating a ledger entry comprising the plurality of hash values and the plurality of location trackers; and

adding the ledger entry to a blockchain ledger.

10. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to:

receive data associated with the computer model;

generate a plurality of asset files based on the data associated with the computer model, wherein generating the plurality of asset files comprises:

generating a first asset file comprising a source data set for the computer model, wherein the source data set comprises a training data set, a testing data set, and a validation data set;

generating a second asset file comprising experimentation data, wherein the experimentation data comprises model development code for the computer model; and

generating a third asset file comprising a binary record representing a model structure and learned functions of the computer model;

generate a plurality of hash values corresponding to the plurality of asset files;

generate a plurality of location trackers corresponding to the plurality of asset files;

generate a ledger entry comprising the plurality of hash values and the plurality of location trackers; and

add the ledger entry to a blockchain ledger.

11. A computer-enabled method for retrieving a computer model, the method comprising:

receiving an input indicative of a computer model;

based on the input indicative of the computer model, obtaining a ledger entry, wherein the ledger entry comprises a first set of one or more hash values, and wherein the ledger entry further comprises one or more location trackers;

obtaining a plurality of asset files based on the one or more location trackers, wherein obtaining the plurality of asset files comprises:

obtaining a first asset file comprising a source data set for the computer model, wherein the source data set comprises a training data set, a testing data set, and a validation data set;

obtaining a second asset file comprising experimentation data, wherein the experimentation data comprises model development code for the computer model; and

obtaining a third asset file comprising a binary record representing a model structure and learned functions of the computer model;

generating a second set of one or more hash values based on the obtained plurality of asset files;

comparing a first hash value of the first set and a second hash value of the second set; and

providing a validation output based on the comparing.

12. The method of claim 11 , wherein the input comprises one or more query terms indicative of the computer model.

13. The method of claim 12 , wherein the obtained ledger entry is associated with one or more keywords matching the one or more query terms in the input.

14. The method of claim 11 , further comprising obtaining an asset file of the one or more asset files by downloading portions of the asset file via a peer-to-peer network and assembling the downloaded portions.

15. The method of claim 11 , further comprising:

determining whether the first hash value is identical to the second hash value;

in accordance with a determination that the first hash value is not identical to the second hash value, providing an output indicating that an asset file corresponding to the first hash value has been altered.

16. An electronic device, comprising:

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

receiving an input indicative of a computer model;

based on the input indicative of the computer model, obtaining a ledger entry, wherein the ledger entry comprises a first set of one or more hash values, and wherein the ledger entry further comprises one or more location trackers;

obtaining a plurality of asset files based on the one or more location trackers, wherein obtaining the plurality of asset files comprises:

obtaining a first asset file comprising a source data set for the computer model, wherein the source data set comprises a training data set, a testing data set, and a validation data set;

obtaining a second asset file comprising experimentation data, wherein the experimentation data comprises model development code for the computer model; and

obtaining a third asset file comprising a binary record representing a model structure and learned functions of the computer model;

generating a second set of one or more hash values based on the obtained plurality of asset files;

comparing a first hash value of the first set and a second hash value of the second set; and

providing a validation output based on the comparing.

17. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to:

receive an input indicative of a computer model;

based on the input indicative of the computer model, obtain a ledger entry, wherein the ledger entry comprises a first set of one or more hash values, and wherein the ledger entry further comprises one or more location trackers;

obtain a plurality of asset files based on the one or more location trackers, wherein obtaining the plurality of asset files comprises:

obtaining a first asset file comprising a source data set for the computer model, wherein the source data set comprises a training data set, a testing data set, and a validation data set;

obtaining a second asset file comprising experimentation data, wherein the experimentation data comprises model development code for the computer model; and

obtaining a third asset file comprising a binary record representing a model structure and learned functions of the computer model;

generate a second set of one or more hash values based on the obtained plurality of asset files;

compare a first hash value of the first set and a second hash value of the second set; and

provide a validation output based on the comparing.

18. A computer-enabled method for providing feedback to a computer model, comprising:

receiving an input indicative of the computer model;

based on the input indicative of the computer model, obtaining a plurality of asset files associated with the computer model using a blockchain ledger, wherein obtaining the plurality of asset files comprises:

obtaining a first asset file comprising a source data set for the computer model, wherein the source data set comprises a training data set, a testing data set, and a validation data set;

obtaining a second asset file comprising experimentation data, wherein the experimentation data comprises model development code for the computer model; and

obtaining a third asset file comprising a binary record representing a model structure and learned functions of the computer model;

receiving an input indicative of a feedback associated with the computer model; and

updating the blockchain ledger based on the feedback.

19. The method of claim 18 , wherein the feedback comprises a performance claim of the computer model.

20. The method of claim 18 , wherein the feedback comprises a disagreement with a performance claim of the computer model, wherein the performance claim of the computer model is in the one or more asset files.

21. An electronic device, comprising:

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

receiving an input indicative of a computer model;

based on the input indicative of the computer model, obtaining a plurality of asset files associated with the computer model using a blockchain ledger, wherein obtaining the plurality of asset files comprises:

obtaining a first asset file comprising a source data set for the computer model, wherein the source data set comprises a training data set, a testing data set, and a validation data set;

obtaining a second asset file comprising experimentation data, wherein the experimentation data comprises model development code for the computer model; and

obtaining a third asset file comprising a binary record representing a model structure and learned functions of the computer model;

receiving an input indicative of a feedback associated with the computer model; and

updating the blockchain ledger based on the feedback.

22. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to:

receive an input indicative of the computer model;

based on the input indicative of the computer model, obtain a plurality of asset files associated with the computer model using a blockchain ledger, wherein obtaining the plurality of asset files comprises:

obtaining a first asset file comprising a source data set for the computer model, wherein the source data set comprises a training data set, a testing data set, and a validation data set;

obtaining a second asset file comprising experimentation data, wherein the experimentation data comprises model development code for the computer model; and

obtaining a third asset file comprising a binary record representing a model structure and learned functions of the computer model;

receive an input indicative of a feedback associated with the computer model; and

update the blockchain ledger based on the feedback.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 9, 2023
From: PRICEWATERHOUSECOOPERS LLP
To: PWC PRODUCT SALES LLC
Reel/Frame 065532/0034 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 24, 2021
From: GOLBIN, ILANA ALEXANDRA; VOYLES, JOSEPH DAVID; KERSEY, KRIS DOUGLAS; FOTH, THOMAS JOSEPH
To: PRICEWATERHOUSECOOPERS LLP
Reel/Frame 057590/0015 →
Continuity (2)
Provisional Application 62990044 · Mar 16, 2020
Related Publication 20210286803A1 · Sep 16, 2021