IP Library Patent Application 14944092
Patent Application
App. No. 14/944,092

MOST IMPACTFUL EXPERIMENTS

Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US None
App. No.
14/944,092
Abstract

Techniques for conducting A/B experimentation of online content are described. According to various embodiments, a user specification of a metric associated with operation of an online social networking service is received. A set of one or more A/B experiments of online content is then identified, each A/B experiment being targeted at a segment of members of the online social networking service. Thereafter, each of the A/B experiments is ranked, based on an inferred impact on the value of the metric in response to application of a treatment variant of each A/B experiment to the online social networking service. A list of one or more of the ranked A/B experiments is then displayed, via a user interface displayed on a client device.

Claims (52)

1 . A method comprising:

receiving a user specification of a metric associated with operation of an online social networking service;

identifying a set of one or more A/B experiments of online content, each A/B experiment being targeted at a segment of members of the online social networking service;

ranking, using one or more hardware processors, each of the A/B experiments, based on an inferred impact on the value of the metric in response to application of a treatment variant of each A/B experiment to the online social networking service; and

displaying, via a user interface displayed on a client device, a list of one or more of the ranked A/B experiments.

2 . The method of claim 1 , wherein the ranking further comprises:

ranking the A/B experiments based at least in part on a site-wide impact value associated with each of the A/B experiments, each site-wide impact value indicating a predicted change in the value of the metric responsive to application of the treatment variant of the A/B experiment to 100 % of a targeted segment of members of the A/B experiment, in comparison to application of a control variant of the A/B experiment to 100 % of the targeted segment of members of the A/B experiment.

3 . The method of claim 1 , wherein the ranking further comprises:

ranking the A/B experiments based at least in part on a ramp percentage value associated with each of the A/B experiments, each ramp percentage value indicating a percentage of the targeted segment of members of the corresponding A/B experiment to which the treatment variant of the corresponding A/B experiment has been applied.

4 . The method of claim 1 , wherein the ranking further comprises:

ranking the A/B experiments based at least in part on an experiment duration value associated with each of the A/B experiments, each experiment duration value indicating a duration of the corresponding A/B experiment.

5 . The method of claim 1 , further comprising:

displaying, via the user interface, a message user interface element associated with each of the A/B experiments in the list;

receiving a user selection of a specific message user interface element associated with a specific one of the A/B experiments in the list; and

automatically generating a draft electronic message addressed to a user registered as the owner of the specific one of the A/B experiments in the list.

6 . The method of claim 1 , wherein the metric is a number of page views associated with a webpage.

7 . The method of claim 1 , wherein the metric is a number of unique visitors associated with a webpage.

8 . The method of claim 1 , wherein the metric is a click-through rate associated with an online content item.

9 . A system comprising:

a processor; and

a memory device holding an instruction set executable on the processor to cause the system to perform operations comprising:

receiving a user specification of a metric associated with operation of an online social networking service;

identifying a set of one or more A/B experiments of online content, each A/B experiment being targeted at a segment of members of the online social networking service;

ranking each of the A/B experiments, based on an inferred impact on the value of the metric in response to application of a treatment variant of each A/B experiment to the online social networking service; and

displaying, via a user interface displayed on a client device, a list of one or more of the ranked A/B experiments.

10 . The system of claim 9 , wherein the ranking further comprises:

ranking the A/B experiments based at least in part on a site-wide impact value associated with each of the A/B experiments, each site-wide impact value indicating a predicted change in the value of the metric responsive to application of the treatment variant of the A/B experiment to 100 % of a targeted segment of members of the A/B experiment, in comparison to application of a control variant of the A/B experiment to 100 % of the targeted segment of members of the A/B experiment.

11 . The system of claim 9 , wherein the ranking further comprises:

ranking the A/B experiments based at least in part on a ramp percentage value associated with each of the A/B experiments, each ramp percentage value indicating a percentage of the targeted segment of members of the corresponding A/B experiment to which the treatment variant of the corresponding A/B experiment has been applied.

12 . The system of claim 9 , wherein the ranking further comprises:

ranking the A/B experiments based at least in part on an experiment duration value associated with each of the A/B experiments, each experiment duration value indicating a duration of the corresponding A/B experiment.

13 . The system of claim 9 , wherein the operations further comprise:

displaying, via the user interface, a message user interface element associated with each of the A/B experiments in the list;

receiving a user selection of a specific message user interface element associated with a specific one of the A/B experiments in the list; and

automatically generating a draft electronic message addressed to a user registered as the owner of the specific one of the A/B experiments in the list.

14 . The system of claim 9 , wherein the metric is a number of page views associated with a webpage.

15 . A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

receiving a user specification of a metric associated with operation of an online social networking service;

identifying a set of one or more A/B experiments of online content, each A/B experiment being targeted at a segment of members of the online social networking service;

ranking each of the A/B experiments, based on an inferred impact on the value of the metric in response to application of a treatment variant of each A/B experiment to the online social networking service; and

displaying, via a user interface displayed on a client device, a list of one or more of the ranked A/B experiments.

16 . The storage medium of claim 15 , wherein the ranking further comprises:

ranking the A/B experiments based at least in part on a site-wide impact value associated with each of the A/B experiments, each site-wide impact value indicating a predicted change in the value of the metric responsive to application of the treatment variant of the A/B experiment to 100 % of a targeted segment of members of the A/B experiment, in comparison to application of a control variant of the A/B experiment to 100 % of the targeted segment of members of the A/B experiment.

17 . The storage medium of claim 15 , wherein the ranking further comprises:

ranking the A/B experiments based at least in part on a ramp percentage value associated with each of the A/B experiments, each ramp percentage value indicating a percentage of the targeted segment of members of the corresponding A/B experiment to which the treatment variant of the corresponding A/B experiment has been applied.

18 . The storage medium of claim 15 , wherein the ranking further comprises:

ranking the A/B experiments based at least in part on an experiment duration value associated with each of the A/B experiments, each experiment duration value indicating a duration of the corresponding A/B experiment.

19 . The storage medium of claim 1 , wherein the operations further comprise:

displaying, via the user interface, a message user interface element associated with each of the A/B experiments in the list;

receiving a user selection of a specific message user interface element associated with a specific one of the A/B experiments in the list; and

automatically generating a draft electronic message addressed to a user registered as the owner of the specific one of the A/B experiments in the list.

20 . The storage medium of claim 15 , wherein the metric is a number of page views associated with a webpage.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 1, 2017
From: LINKEDIN CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 044746/0001 →
CORRECTIVE ASSIGNMENT TO CORRECT THE SPELLING OF INVENTOR JARAMILLO'S NAME PREVIOUSLY RECORDED AT REEL: 037889 FRAME: 0758. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT . Recorded May 2, 2016
From: XU, YA; SINNO, OMAR; FERNANDEZ, ADRIAN AXEL REMIGO; CHEN, NANYU; LOPUS, CHRISTINA LYNN; CHEN, BRYAN TAI AN; NIEH, KYLAN MATTHEW; HURTADO JARAMILLO, LUISA FERNANDA; BING, JIE
To: LINKEDIN CORPORATION
Reel/Frame 038598/0286 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 4, 2016
From: XU, YA; SINNO, OMAR; FERNANDEZ, ADRIAN AXEL REMIGO; CHEN, NANYU; LOPUS, CHRISTINA LYNN; CHEN, BRYAN TAI AN; NIEH, KYLAN MATTHEW; JARAMILLO, LUISA FERNANDA HURTADO; BING, JIE
To: LINKEDIN CORPORATION
Reel/Frame 037889/0758 →