IP Library › Granted Patent US 10,878,339
Granted Patent B2
US 10,878,339 · App. 15/417,459 · Granted Dec 29, 2020

Leveraging machine learning to predict user generated content

Inventors: Arun Mathew (New York, NY); Kaleigh Smith (Brooklyn, NY); Per Anderson (New York, NY); Ian Langmore (Brooklyn, NY)
Assignee: Google LLC
G06N20/00G06N3/04G06N5/04
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,878,339
App. No.
15/417,459
Granted
Dec 29, 2020
Kind
B2
Abstract

Systems and methods of leveraging machine learning to predict user generated content are provided. For instance, first entity data associated with an entity can be received. The first entity data can include user specified data associated with an attribute of the entity. The first entity data can be input into a machine-learned content prediction model. Inferred entity data can be received as output of the machine-learned content prediction model. The inferred entity data can include inferred data descriptive of the attribute of the entity.

Claims (38)

1. A computer-implemented method of predicting user generated content, the method comprising:

receiving, by one or more computing devices, first entity data and global entity data, wherein the first entity data comprises one or more responses to an information collection task associated with whether or not an entity possesses a particular attribute, and wherein the global entity data is associated with user responses to other information collection tasks associated with the entity;

inputting, by the one or more computing devices, the first entity data and the global entity data into a machine-learned content prediction model;

receiving as output of the machine-learned content prediction model, by the one or more computing devices, inferred entity data comprising a confidence score based on an estimated response rate-associated with a predicted ratio of the one or more responses to the information collection task that are true, wherein whether or not the entity possesses the particular attribute is based at least in part on whether or not the estimated response rate exceeds a response rate threshold; and

determining, by the one or more computing devices, a utility score based at least in part on the inferred entity data, wherein the utility score indicates an estimated increase in a likelihood that the estimated response rate will exceed the response rate threshold upon receiving an additional response to the information collection task.

2. The computer-implemented method of claim 1 , wherein the information collection task comprises a question associated with the particular attribute of the entity, and wherein the first entity data comprises one or more answers to the question provided by the one or more users.

3. The computer-implemented method of claim 2 , wherein the machine-learned content prediction model further comprises a logistic regression and a beta-binomial model coupled to the logistic regression.

4. The computer-implemented method of claim 3 , further comprising:

receiving, by the one or more computing devices, the estimated response rate to the information collection task as output of the logistic regression; and

inputting, by the one or more computing devices, the estimated response rate to the beta-binomial model of the machine-learned content prediction model.

5. The computer-implemented method of claim 4 , wherein receiving as output of the machine-learned content prediction model, by the one or more computing devices, inferred entity data comprises receiving the inferred entity data as output of the beta-binomial model.

6. The computer-implemented method of claim 1 , wherein the inferred entity data comprises an estimated collective result of the information collection task, the estimated collective result determined based at least in part on the estimated response rate and the confidence score.

7. The computer-implemented method of claim 1 , further comprising determining, by the one or more computing devices, a prioritization of a plurality of information collection tasks based at least in part on the utility score.

8. The computer-implemented method of claim 7 , further comprising providing, by the one or more computing devices, one or more additional information collection tasks to one or more users based at least in part on the prioritization.

9. The computer-implemented method of claim 1 , further comprising training, by the one or more computing devices; the content prediction model based on a set of training data;

wherein training the content prediction model comprises backpropagating, by the one or more computing devices, a loss function through the content prediction model.

10. The computer-implemented method of claim 1 , wherein the machine-learned content prediction model comprises a neural network that integrates a beta-binomial loss function.

11. A computing system, comprising:

one or more processors; and

one or more memory devices, the one or more memory devices storing computer-readable instructions that when executed by the one or more processors cause the one or more processors to perform operations, the operations comprising:

receiving first entity data and global entity data, wherein the first entity data comprises one or more responses to an information collection task associated with whether or not an entity possesses a particular attribute, and wherein the global entity data is associated with user responses to other information collection tasks associated with the entity;

inputting the first entity data and the global entity data into a machine-learned content prediction model;

receiving as output of the machine-learned content prediction model, inferred entity data comprising a confidence score based on an estimated response rate associated with a predicted ratio of the one or more responses to the information collection task that are true, wherein whether or not the entity possesses the particular attribute is based at least in part on whether or not the estimated response rate exceeds a response rate threshold; and

determining a utility score based at least in part on the inferred entity data, wherein the utility score indicates an estimated increase in a likelihood that the estimated response rate will exceed the response rate threshold upon receiving an additional response to the information collection task.

12. The computing system of claim 11 , wherein the information collection task comprises a question associated with the particular attribute of the entity, and wherein the first entity data comprises one or more answers to the question provided by the one or more users.

13. The computing system of claim 11 , wherein the machine-learned content prediction model further comprises a logistic regression and a beta-binomial model coupled to the logistic regression.

14. One or more tangible, non-transitory computer-readable media storing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform operations, the operations comprising:

receiving first entity data and global entity data, wherein the first entity data comprises one or more responses to an information collection task associated with whether or not an entity possesses a particular attribute, and wherein the global entity data is associated with user responses to other information collection tasks associated with the entity;

inputting the first entity data and the global entity data into a machine-learned content prediction model;

receiving as output of the machine-learned content prediction model, inferred entity data comprising a confidence score based on an estimated response rate associated with a predicted ratio of the one or more responses to the information collection task that are true, wherein whether or not the entity possesses the particular attribute is based at least in part on whether or not the estimated response rate exceeds a response rate threshold; and

determining a utility score based at least in part on the inferred entity data, wherein the utility score indicates an estimated increase in a likelihood that the estimated response rate will exceed the response rate threshold upon receiving an additional response to the information collection task.

15. The one or more tangible, non-transitory computer-readable media of claim 14 , wherein the machine-learned content prediction model further comprises a logistic regression and a beta-binomial model coupled to the logistic regression.

16. The one or more tangible, non-transitory computer-readable media of claim 14 , wherein the machine-learned content prediction model comprises a neural network that integrates a beta-binomial loss function.

17. The one or more tangible, non-transitory computer-readable media of claim 14 , wherein the machine-learned content prediction model is configured to generate a probability density function specifying probabilities associated with the information collection task.

18. The computer-implemented method of claim 1 , wherein the machine-learned content prediction model is configured to generate a probability density function specifying probabilities associated with the information collection task.

19. The computing system of claim 11 , wherein the machine-learned content prediction model is configured to generate a probability density function specifying probabilities associated with the information collection task.

20. The computing system of claim 11 , further comprising:

determining a prioritization of a plurality of information collection tasks based at least in on the utility score.

Assignments (2)
CHANGE OF NAME Recorded Oct 20, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044567/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 27, 2017
From: MATHEW, ARUN; SMITH, KALEIGH; ANDERSON, PER; LANGMORE, IAN
To: GOOGLE INC.
Reel/Frame 041101/0955 →
Continuity (1)
Related Publication 20180218282A1 · Aug 2, 2018
Cited By (1)
US 12,711,117