IP Library › Granted Patent US 11,164,107
Granted Patent B1
US 11,164,107 · App. 15/937,227 · Granted Nov 2, 2021

Apparatuses and methods for evaluation of proffered machine intelligence in predictive modelling using cryptographic token staking

Inventors: Richard Malone Craib (San Francisco, CA); Geoffrey Bradway (San Francisco, CA); Alexander Dunn (San Francisco, CA)
Assignee: NUMERAI, INC.
G06N20/00G06N5/04H04L67/104
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,164,107
App. No.
15/937,227
Filed
Mar 27, 2018
Granted
Nov 2, 2021
Kind
B1
Art Unit
2456
USPC
706/10
Abstract

A method of improving performance of machine learning models by leveraging crowdsourced artificial intelligence includes sending first data to a plurality of data source compute nodes and receiving indications of stakes and estimates based on the first data from the plurality of data source compute nodes. Each data source compute node is ranked based on the received indications of stakes, to generate a plurality of ranked data source compute nodes. An accuracy of each received estimate is calculated by comparing the received estimates to second data. Until a predefined resource is depleted, and in order of rank, if the accuracy of the estimate associated with a ranked data source compute node exceeds a predefined threshold, the predefined resource is decremented and a token augmentation can be assigned to the ranked data source compute node.

Claims (70)

1. A processor-implemented method, comprising:

generating, at a host system, first data including encrypted data that does not include any information that is proprietary to the host system;

sending the first data from the host system to each data source compute node of a plurality of data source compute nodes, in response to download requests received from each data source compute node of the plurality of data source compute nodes;

receiving, via an application programming interface (“API”) of the host system and from each data source compute node of the plurality of data source compute nodes, an estimate based on the first data;

receiving an indication of a stake associated with each data source compute node of the plurality of data source compute nodes;

storing, in memory, an indication of a predefined feedback resource associated with a smart contract;

ranking each data source compute node of the plurality of data source compute nodes based on the received indications of stakes, to generate a plurality of ranked data source compute nodes;

calculating, via at least one processor of the host system, an accuracy of each received estimate by comparing the received estimates to second data;

for a first ranked data source compute node from the plurality of ranked data source compute nodes:

decrementing the predefined feedback resource and assigning a token augmentation to the first ranked data source compute node if the accuracy of the estimate associated with the first ranked data source compute node has a log loss of less than about −ln(0.5); and

for each remaining ranked data source compute node from the plurality of ranked data source compute nodes:

determining a value associated with the predefined feedback resource;

if the value is greater than zero, decrementing the predefined feedback resource and assigning a token augmentation to the ranked data source compute node if the accuracy of the estimate associated with the ranked data source compute node exceeds a predefined threshold.

2. A method, comprising:

sending first data to each data source compute node of a plurality of data source compute nodes;

receiving, from each data source compute node of the plurality of data source compute nodes, an estimate based on the first data;

receiving an indication of a stake associated with each data source compute node of the plurality of data source compute nodes;

storing, in memory, an indication of a predefined resource;

ranking each data source compute node of the plurality of data source compute nodes based on the received indications of stakes, to generate a plurality of ranked data source compute nodes;

calculating, via a processor, an accuracy of each received estimate by comparing the received estimates to second data;

for a first ranked data source compute node from the plurality of ranked data source compute nodes:

decrementing the predefined resource and assigning a token augmentation to the first ranked data source compute node if the accuracy of the estimate associated with the first ranked data source compute node exceeds a predefined threshold; and

for each remaining ranked data source compute node from the plurality of ranked data source compute nodes:

determining a value associated with the predefined resource;

if the value is greater than zero, decrementing the predefined resource and assigning a token augmentation to the ranked data source compute node if the accuracy of the estimate associated with the ranked data source compute node exceeds the predefined threshold.

3. The method of claim 2 , further comprising receiving, from each data source compute node of the plurality of data source compute nodes, an indication of a confidence level wherein the ranking each data source compute node of the plurality of data source compute nodes is further based on the received indications of confidence level.

4. The method of claim 2 , wherein the predefined resource is defined by a smart contract, and decrementing the predefined resource includes one of modifying and replacing the smart contract.

5. The method of claim 2 , wherein receiving the estimates from each data source compute node of the plurality of data source compute nodes is via an application programming interface (“API”).

6. The method of claim 2 , wherein the estimate associated with the first ranked data source compute node exceeds a predefined threshold if log loss<−ln(0.5).

7. The method of claim 2 , further comprising calculating, via the processor, a consistency of each received estimate,

wherein decrementing the predefined resource and assigning the token augmentation to the first ranked data source compute node is further predicated on the estimate associated with the first ranked data source compute node having a consistency of at least 75%.

8. A method, comprising:

sending first data from a host node to each data source compute node of a plurality of data source compute nodes, the first data including target data;

receiving, at the host node, an indication of a stake associated with each data source compute node of the plurality of data source compute nodes;

sending, in response to receiving the indications of stake, a stake identifier to each data source compute node of the plurality of data source compute nodes;

receiving, at the host node and from each data source compute node of the plurality of data source compute nodes, a target estimate based on the first data and associated with the target data; and

calculating, at the host node and for each received target estimate, a predictive accuracy value associated with the target estimate;

storing the calculated predictive accuracy values associated with the target estimates;

for a first data source compute node of the plurality of data source compute nodes:

decrementing a predefined divisible resource and assigning a value augmentation to the first data source compute node if the predictive accuracy value associated with the first data source compute node exceeds a predefined threshold; and

for each remaining data source compute node of the plurality of data source compute nodes:

determining a value associated with the predefined divisible resource, and

if the value associated with the predefined divisible resource is greater than zero, decrementing the predefined divisible resource and assigning a value to the data source compute node if the predictive accuracy value associated with the data source compute node exceeds the predefined threshold.

9. The method of claim 8 , wherein the first data includes target data, feature data, and identifier data.

10. The method of claim 9 , wherein the data further includes at least one of: era data or indications of data type.

11. The method of claim 8 , wherein the sending the first data to a first data source compute node of the plurality of data source compute nodes is in response to a download request received from the first data source compute node.

12. The method of claim 8 , wherein the sending the first data is via an application programming interface (“API”).

13. The method of claim 8 , further comprising calculating, at the host node and for each received target estimate, a consistency value associated with the target estimate.

14. The method of claim 8 , further comprising calculating, at the host node and for each received target estimate, an originality value associated with the target estimate.

15. The method of claim 8 , further comprising calculating, at the host node and for each received target estimate, a concordance value associated with the target estimate.

16. The method of claim 8 , further comprising:

calculating, at the host node and for each received target estimate, a consistency value associated with the target estimate, an originality value associated with the target estimate and a concordance value associated with the target estimate; and

sending a signal to cause the calculated predictive accuracy values, the consistency values, and the originality values to be viewable to each data source compute node of the plurality of data source compute nodes.

17. The method of claim 8 , further comprising:

sending, in response to receiving the indications of stake, and to each data source compute node of the plurality of data source compute nodes, at least one of: a stake status, a transaction hash, or an indication of a stake value.

18. A method, comprising:

storing, in memory, an indication of a predefined divisible resource;

ranking each data source compute node of a plurality of data source compute nodes based on indications of stakes associated with each data source compute node of the plurality of data source compute nodes, to generate a plurality of ranked data source compute nodes;

calculating, via a processor and for each data source compute node of the plurality of data source compute nodes, an accuracy of a target probability estimate received from the data source compute node by comparing the received target probability estimate to a stored dataset;

for a first ranked data source compute node of the plurality of ranked data source compute nodes:

decrementing the predefined divisible resource and assigning a value augmentation to the first ranked data source compute node if the accuracy of the target probability estimate associated with the first ranked data source compute node exceeds a predefined threshold; and

for each remaining ranked data source compute node of the plurality of ranked data source compute nodes:

determining a value associated with the predefined divisible resource;

if the value is greater than zero, decrementing the predefined divisible resource and assigning a value augmentation to the ranked data source compute node if the accuracy of the target probability estimate associated with the ranked data source compute node exceeds the predefined threshold.

19. The method of claim 18 , wherein the first ranked data source compute node exceeds the predefined threshold if log loss<−ln(0.5).

20. The method of claim 18 , further comprising:

calculating, for a data source compute node of the plurality of data source compute nodes, a consistency value associated with the target probability estimate received from that data source compute node; and

excluding that data source compute node from the plurality of ranked data source compute nodes if the consistency value is below 75%.

21. The method of claim 18 , wherein ranking the data source compute nodes is further based on indications of confidence received from each data source compute node of the plurality of data source compute nodes.

22. The method of claim 18 , wherein the value augmentation has a value of s/c, where “s” is associated with the indication of stake, and “c” is associated with the indication of confidence.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 9, 2020
From: NUMERAI GP LLC
To: NUMERAI, INC.
Reel/Frame 053167/0237 →
CHANGE OF NAME Recorded Aug 14, 2019
From: NUMERAI, LLC
To: NUMERAI GP LLC
Reel/Frame 050054/0110 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 2, 2018
From: CRAIB, RICHARD M.; BRADWAY, GEOFFREY; DUNN, ALEXANDER
To: NUMERAI, LLC
Reel/Frame 045407/0936 →
Continuity (2)
Provisional Application 62477235 · Mar 27, 2017
Provisional Application 62648595 · Mar 27, 2018
Cited By (4)
US 12,301,575 US 12,346,432 US 12,574,263 US 12,688,456