IP Library Granted Patent US 10,839,418
Granted Patent B1
US 10,839,418 · App. 16/200,606 · Granted Nov 17, 2020

Predicting performance of content item campaigns

Inventors: Prathyusha Aragonda (Mountain View, CA); Sumeet Kumar (Belmont, CA); Spencer Bingham Powell (San Francisco, CA)
Assignee: Facebook, Inc.
G06Q30/0243G06Q30/0249
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,839,418
App. No.
16/200,606
Granted
Nov 17, 2020
Kind
B1
Abstract

An online system predicts a performance of a content item campaign based on a set of received target parameters from a content item publisher and compares the predicted performance of the content item campaign to predicted performances of comparison campaigns. The comparison campaigns are selected by performing a plurality of simulations on prior content item campaigns to estimate campaign presentation results for each prior content item campaign if presented to users with the sets of test campaign parameters. After generating histograms based on the estimated results, the online system selects prior content item campaigns for the set of comparison campaigns based on changes in histogram positions of the prior content item campaigns. The online system generates a set of estimated campaign presentation results for the content item campaign and the set of comparison campaigns for comparison.

Claims (60)

1. A method comprising:

identifying, by an online system, a set of prior content item campaigns presented to users of an online system;

identifying, by the online system, a plurality of sets of test campaign parameters, each set of test campaign parameters describing a different combination of parameters for presenting content to users;

selecting, by the online system, a set of comparison campaigns from the set of prior content item campaigns by:

for each set of test campaign parameters in the plurality of sets of test campaign parameters:

estimating campaign presentation results for each prior content item campaign if presented to users with the set of test campaign parameters;

identifying a score of the prior content item campaigns according to the estimated campaign presentation results; and

generating a histogram of the prior content item campaigns that assigns a bin to a prior content item campaign based on the score;

identifying, by the online system, a change in histogram position for each prior content item campaign across the histograms associated with the set of test campaign parameters; and

selecting the set of comparison campaigns based on the change in histogram position for the prior content item campaigns;

receiving a request from an entity, by the online system, to determine an estimated campaign presentation result for a content item campaign associated with the entity, the request including a set of target parameters for the content item campaign;

responsive to receiving the request, generating, by the online system, the estimated campaign presentation result for the content item campaign and a set of estimated campaign presentation results for the set of comparison campaigns in real-time by estimating performances of the content item campaign and the set of comparison campaigns if presented with the set of target parameters; and

presenting the estimated campaign presentation result for the content item campaign and the set of estimated campaign presentation results for the set of comparison campaigns to the entity on a graphical user interface (GUI).

2. The method of claim 1 , wherein the set of comparison campaigns are selected from the prior content item campaigns having a change in histogram position of one or less across the histograms associated with the set of test campaign parameters.

3. The method of claim 1 , further comprising filtering the set of prior content item campaigns to determine a filtered set of prior content item campaigns, wherein the set of comparison campaigns is selected from the filtered set of prior content item campaigns and the prior content item campaigns are filtered to exclude content item campaigns based on a total value of the campaign, a reach of a campaign, a type of targeting criteria, and an audience size of the campaign.

4. The method of claim 1 , wherein the content item campaign has a desired interaction type, and the set of prior content item campaigns excludes prior content item campaigns having a different desired interaction type.

5. The method of claim 1 , wherein the set of comparison campaigns is selected from the set of prior content item campaigns to include a diversity of average histogram positions.

6. The method of claim 1 , wherein the prior content item campaigns each have an associated set of delivery parameters and were presented to users according to the set of delivery parameters; and further wherein the estimated campaign presentation results for each prior content item campaign is based on the results of presenting content to users of the campaign according to the delivery parameters for the prior content item campaign.

7. The method of claim 1 , wherein the estimated campaign presentation results include estimated interaction rates and are presented to the entity for comparison with a historical interaction rate for the entity based on prior content item campaigns of the entity.

8. A non-transitory computer-readable medium comprising computer program instructions that when executed by a computer processor of an online system cause the processor to perform steps comprising:

identifying a set of prior content item campaigns presented to users of an online system;

identifying a plurality of sets of test campaign parameters, each set of test campaign parameters describing a different combination of parameters for presenting content to users;

selecting a set of comparison campaigns from the prior content item campaigns by:

for each set of test campaign parameters in the set of test campaign parameters:

estimating campaign presentation results for each prior content item campaign if presented to users with the set of test campaign parameters;

identifying a score of the prior content item campaigns according to the estimated campaign presentation results; and

generating a histogram of the prior content item campaigns that assigns a bin to a prior content item campaign based the score;

identifying a change in histogram position for each prior content item campaign across the histograms associated with the set of test campaign parameters; and

selecting the set of comparison campaigns based on the change in histogram position for the prior content item campaigns;

receiving a request from an entity to determine an estimated campaign presentation result for a content item campaign associated with the entity, the request including a set of target parameters for the content item campaign;

responsive to receiving the request, generating the estimated campaign presentation result for the content item campaign and a set of estimated campaign presentation results for the set of comparison campaigns in real-time by estimating performances of the content item campaign and the set of comparison campaigns if presented with the set of target parameters; and

presenting the estimated campaign presentation result for the content item campaign and the set of estimated campaign presentation results for the set of comparison campaigns to the entity on a graphical user interface (GUI).

9. The non-transitory computer-readable medium of claim 8 , wherein the set of comparison campaigns are selected from the prior content item campaigns having a change in histogram position of one or less across the histograms associated with the set of test campaign parameters.

10. The non-transitory computer-readable medium of claim 8 , wherein the non-transitory computer-readable medium further comprises instructions that when executed by a computer processor of an online system cause the processor to determine a filtered set of prior content item campaigns, wherein the set of comparison campaigns is selected from the filtered set of prior content item campaigns and the prior content item campaigns are filtered to exclude content item campaigns based on a total value of the campaign, a reach of a campaign, a type of targeting criteria, and an audience size of the campaign.

11. The non-transitory computer-readable medium of claim 8 , wherein the subject campaign has a desired interaction type, and the set of prior content item campaigns excludes prior content item campaigns having a different desired interaction type.

12. The non-transitory computer-readable medium of claim 8 , wherein the set of comparison campaigns is selected from the set of prior content item campaigns to include a diversity of average histogram positions.

13. The non-transitory computer-readable medium of claim 8 , wherein the prior content item campaigns each have an associated set of delivery parameters and were presented to users according to the set of delivery parameters; and further wherein the estimated campaign presentation results for each prior content item campaign is based on the results of presenting content to users of the campaign according to the delivery parameters for the prior content item campaign.

14. The non-transitory computer-readable medium of claim 8 , wherein the estimated campaign presentation results include estimated interaction rates and are presented to the entity for comparison with a historical interaction rate for the entity.

15. A system comprising:

a processor; and

a non-transitory computer-readable medium comprising computer program instructions that when executed by the processor of an online system causes the processor to perform steps comprising:

identifying a set of prior content item campaigns presented to users of an online system;

identifying a plurality of sets of test campaign parameters, each set of test campaign parameters describing a different combination of parameters for presenting content to users;

selecting a set of comparison campaigns from the prior content item campaigns by:

for each set of test campaign parameters in the set of test campaign parameters:

estimating campaign presentation results for each prior content item campaign if presented to users with the set of test campaign parameters;

identifying a score of the prior content item campaigns according to the estimated campaign presentation results; and

generating a histogram of the prior content item campaigns that assigns a bin to a prior content item campaign based the score;

identifying a change in histogram position for each prior content item campaign across the histograms associated with the set of test campaign parameters; and

selecting the set of comparison campaigns based on the change in histogram position for the prior content item campaigns;

receiving a request from an entity to determine an estimated campaign presentation result for a content item campaign associated with the entity, the request including a set of target parameters for the content item campaign;

responsive to receiving the request, generating the estimated campaign presentation result for the content item campaign and a set of estimated campaign presentation results for the set of comparison campaigns in real-time by estimating performances of the content item campaign and the set of comparison campaigns if presented with the set of target parameters; and

presenting the estimated campaign presentation result for the content item campaign and the set of estimated campaign presentation results for the set of comparison campaigns to the entity on a graphical user interface (GUI).

16. The system of claim 15 , wherein the set of comparison campaigns are selected from the prior content item campaigns having a change in histogram position of one or less across the histograms associated with the set of test campaign parameters.

17. The system of claim 15 , wherein the non-transitory computer-readable medium comprises computer program instructions that when executed by the processor of the online system causes the processor to perform steps further comprising:

filtering the set of prior content item campaigns to determine a filtered set of prior content item campaigns, wherein the set of comparison campaigns is selected from the filtered set of prior content item campaigns and the prior content item campaigns are filtered to exclude content item campaigns based on a total value of the campaign, a reach of a campaign, a type of targeting criteria, and an audience size of the campaign.

18. The system of claim 15 wherein the set of comparison campaigns are selected from the prior content item campaigns having a change in histogram position of one or less across the histogram associated with the set of test campaign parameters.

19. The system of claim 15 , wherein the non-transitory computer-readable medium comprises computer program instructions that when executed by the processor of the online system causes the processor to perform steps further comprising:

filtering the set of prior content item campaigns to determine a filtered set of prior content item campaigns, wherein the set of comparison campaigns is selected from the filtered set of prior content item campaigns and the prior content item campaigns are filtered to exclude content item campaigns based on a total value of the campaign, a reach of a campaign, a type of targeting criteria, and an audience size of the campaign.

20. The system of claim 15 , wherein the content item campaign has a desired interaction type, and the set of prior content item campaigns exclude prior content item campaigns having a different desired interaction type.

Assignments (2)
CHANGE OF NAME Recorded Nov 18, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058897/0824 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 11, 2018
From: ARAGONDA, PRATHYUSHA; KUMAR, SUMEET; POWELL, SPENCER BINGHAM
To: FACEBOOK, INC.
Reel/Frame 047739/0553 →
Cited By (3)
US 12,401,575 US 12,411,893 US 12,555,290