PREDICTING AN EFFECT OF A SET OF MODIFICATIONS TO AN APPEARANCE OF CONTENT INCLUDED IN A CONTENT ITEM ON A PERFORMANCE METRIC ASSOCIATED WITH THE CONTENT ITEM

1 . A method comprising:

receiving a request from a content-providing user of an online system to generate a content item to be presented to one or more viewing users of the online system, the request specifying one or more modifications to an appearance of content received from the content-providing user;

generating a plurality of content item instances of the content item, each of the plurality of content item instances including a different set of the one or more modifications to the appearance of the content specified in the request;

generating an identifier for each of the plurality of content item instances, each identifier associated with the set of the one or more modifications to the appearance of the content included in the content item instance;

presenting the plurality of content item instances to a subset of the one or more viewing users of the online system;

tracking a performance metric associated with impressions of each of the plurality of content item instances using the identifier associated with the set of the one or more modifications to the appearance of the content included in each of the plurality of content item instances;

identifying one or more pairs of the plurality of content item instances; and

for each of the one or more pairs of the plurality of content item instances:

comparing a first value of the performance metric associated with a first content item instance of the pair to a second value of the performance metric associated with a second content item instance of the pair;

determining a difference between the first value of the performance metric associated with the first content item instance and the second value of the performance metric associated with the second content item instance based at least in part on the comparing;

identifying a subset of the one or more modifications to the appearance of the content to which the difference between the first value of the performance metric associated with the first content item instance and the second value of the performance metric associated with the second content item instance is attributable; and

predicting an improvement in a value of the performance metric associated with content item instances including the subset of the one or more modifications, based at least in part on the difference between the first value of the performance metric associated with the first content item instance and the second value of the performance metric associated with the second content item instance.

2 . The method of claim 1 , wherein the identifier associated with the set of the one or more modifications to the appearance of the content included in each of the plurality of content item instances comprises a digital watermark, an image fingerprint, or an image hash.

3 . The method of claim 1 , wherein the one or more modifications to the appearance of the content are selected from a group consisting of: modifying one or more colors of the content, modifying a placement of an element of the content, modifying a size of the content, modifying a size of an element of the content, modifying a color of an element of the content, and any combination thereof.

4 . The method of claim 1 , wherein the improvement in the value of the performance metric associated with content item instances including the subset of the one or more modifications is predicted by a machine-learned model.

5 . The method of claim 1 , wherein the request is received from the content-providing user via a tool provided by the online system.

6 . The method of claim 5 , wherein the one or more modifications are specified using one or more features of the tool.

7 . The method of claim 6 , further comprising:

identifying a feature of the tool used to specify the subset of the one or more modifications to the appearance of the content to which the difference between the first value of the performance metric associated with the first content item instance and the second value of the performance metric associated with the second content item instance is attributable; and

predicting the improvement in the value of the performance metric associated with content item instances including the subset of the one or more modifications based at least in part on the difference between the first value of the performance metric associated with the first content item instance and the second value of the performance metric associated with the second content item instance.

8 . The method of claim 1 , wherein the content comprises one or more selected from a group consisting of: an image, a photograph, text, and any combination thereof.

9 . The method of claim 1 , wherein the performance metric describes a number of times a content item instance is accessed, a number of times a preference for the content item instance is indicated, a number of installations of an application associated with the content item instance, a number of times an application associated with the content item instance is accessed, a number of purchases of a product associated with the content item instance, a number of purchases of a service associated with the content item instance, a number of views of data associated with the content item instance, a number of conversions associated with the content item instance, a number of subscriptions associated with the content item instance, or a number of interactions with the content item instance.

10 . The method of claim 1 , further comprising:

ranking the plurality of content item instances based at least in part on the value of the performance metric associated with each content item instance of the plurality of content item instances;

determining an amount of variation in values of the performance metric associated with the plurality of content item instances;

responsive to determining the amount of variation in values of the performance metric associated with the plurality of content item instances is at least a threshold amount, identifying an additional subset of the one or more modifications to the appearance of the content to which the amount of variation in values of the performance metric associated with the plurality of content item instances is attributable; and

predicting the improvement in the value of the performance metric associated with content item instances including the additional subset of the one or more modifications based at least in part on the ranking and the amount of variation in values of the performance metric associated with the plurality of content item instances.

11 . The method of claim 1 , further comprising:

storing the identifier associated with the set of the one or more modifications to the appearance of the content included in each of the plurality of content item instances in association with each of the plurality of content item instances including the set of the one or more modifications to the appearance of the content.

12 . The method of claim 1 , further comprising:

communicating the predicted improvement in the value of the performance metric associated with content item instances including the subset of the one or more modifications to the content-providing user of the online system.

13 . The method of claim 1 , further comprising:

receiving the content from the content-providing user of the online system.

14 . A computer program product comprising a computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to:

receive a request from a content-providing user of an online system to generate a content item to be presented to one or more viewing users of the online system, the request specifying one or more modifications to an appearance of content received from the content-providing user;

generate a plurality of content item instances of the content item, each of the plurality of content item instances including a different set of the one or more modifications to the appearance of the content specified in the request;

generate an identifier for each of the plurality of content item instances, each identifier associated with the set of the one or more modifications to the appearance of the content included in the content item instance;

present the plurality of content item instances to a subset of the one or more viewing users of the online system;

track a performance metric associated with impressions of each of the plurality of content item instances using the identifier associated with the set of the one or more modifications to the appearance of the content included in each of the plurality of content item instances;

identify one or more pairs of the plurality of content item instances; and

for each of the one or more pairs of the plurality of content item instances:

compare a first value of the performance metric associated with a first content item instance of the pair to a second value of the performance metric associated with a second content item instance of the pair;

determine a difference between the first value of the performance metric associated with the first content item instance and the second value of the performance metric associated with the second content item instance based at least in part on the comparing;

identify a subset of the one or more modifications to the appearance of the content to which the difference between the first value of the performance metric associated with the first content item instance and the second value of the performance metric associated with the second content item instance is attributable; and

predict an improvement in a value of the performance metric associated with content item instances including the subset of the one or more modifications, based at least in part on the difference between the first value of the performance metric associated with the first content item instance and the second value of the performance metric associated with the second content item instance.

15 . The computer program product of claim 14 , wherein the identifier associated with the set of the one or more modifications to the appearance of the content included in each of the plurality of content item instances comprises a digital watermark, an image fingerprint, or an image hash.

16 . The computer program product of claim 14 , wherein the one or more modifications to the appearance of the content are selected from a group consisting of:

modifying one or more colors of the content, modifying a placement of an element of the content, modifying a size of the content, modifying a size of an element of the content, modifying a color of an element of the content, and any combination thereof.

17 . The computer program product of claim 14 , wherein the improvement in the value of the performance metric associated with content item instances including the subset of the one or more modifications is predicted by a machine-learned model.

18 . The computer program product of claim 14 , wherein the request is received from the content-providing user via a tool provided by the online system.

19 . The computer program product of claim 18 , wherein the one or more modifications are specified using one or more features of the tool.

20 . The computer program product of claim 18 , wherein the computer readable storage medium further has instructions encoded thereon that, when executed by the processor, cause the processor to:

identifying a feature of the tool used to specify the subset of the one or more modifications to the appearance of the content to which the difference between the first value of the performance metric associated with the first content item instance and the second value of the performance metric associated with the second content item instance is attributable; and

predicting the improvement in the value of the performance metric associated with content item instances including the subset of the one or more modifications based at least in part on the difference between the first value of the performance metric associated with the first content item instance and the second value of the performance metric associated with the second content item instance.

21 . The computer program product of claim 14 , wherein the content comprises one or more selected from a group consisting of: an image, a photograph, text, and any combination thereof.

22 . The computer program product of claim 14 , wherein the performance metric describes a number of times a content item instance is accessed, a number of times a preference for the content item instance is indicated, a number of installations of an application associated with the content item instance, a number of times an application associated with the content item instance is accessed, a number of purchases of a product associated with the content item instance, a number of purchases of a service associated with the content item instance, a number of views of data associated with the content item instance, a number of conversions associated with the content item instance, a number of subscriptions associated with the content item instance, or a number of interactions with the content item instance.

23 . The computer program product of claim 14 , wherein the computer readable storage medium further has instructions encoded thereon that, when executed by the processor, cause the processor to:

rank the plurality of content item instances based at least in part on the value of the performance metric associated with each content item instance of the plurality of content item instances;

determine an amount of variation in values of the performance metric associated with the plurality of content item instances;

responsive to determine the amount of variation in values of the performance metric associated with the plurality of content item instances is at least a threshold amount, identify an additional subset of the one or more modifications to the appearance of the content to which the amount of variation in values of the performance metric associated with the plurality of content item instances is attributable; and

predict the improvement in the value of the performance metric associated with content item instances including the additional subset of the one or more modifications based at least in part on the ranking and the amount of variation in values of the performance metric associated with the plurality of content item instances.

24 . The computer program product of claim 14 , wherein the computer readable storage medium further has instructions encoded thereon that, when executed by the processor, cause the processor to:

store the identifier associated with the set of the one or more modifications to the appearance of the content included in each of the plurality of content item instances in association with each of the plurality of content item instances including the set of the one or more modifications to the appearance of the content.

25 . The computer program product of claim 14 , wherein the computer readable storage medium further has instructions encoded thereon that, when executed by the processor, cause the processor to:

communicate the predicted improvement in the value of the performance metric associated with content item instances including the subset of the one or more modifications to the content-providing user of the online system.

26 . The computer program product of claim 14 , wherein the computer readable storage medium further has instructions encoded thereon that, when executed by the processor, cause the processor to:

receive the content from the content-providing user of the online system.

27 . A method comprising:

receiving a request from a content-providing user of an online system to generate a content item to be presented to one or more viewing users of the online system, the request specifying one or more modifications to an appearance of content received from the content-providing user;

generating a plurality of content item instances of the content item, each of the plurality of content item instances including a different set of the one or more modifications to the appearance of the content specified in the request;

generating an identifier for each of the plurality of content item instances, each identifier associated with the set of the one or more modifications to the appearance of the content included in the content item instance;

presenting the plurality of content item instances to a subset of the one or more viewing users of the online system;

tracking a performance metric associated with impressions of each of the plurality of content item instances using the identifier associated with the set of the one or more modifications to the appearance of the content included in each of the plurality of content item instances;

ranking the plurality of content item instances based at least in part on a value of the performance metric associated with each content item instance of the plurality of content item instances;

determining an amount of variation in values of the performance metric associated with the plurality of content item instances;

responsive to determining the amount of variation in values of the performance metric associated with the plurality of content item instances is at least a threshold amount, identifying a subset of the one or more modifications to the content to which the amount of variation in values of the performance metric associated with the plurality of content item instances is attributable; and

predicting an effect on the value of the performance metric associated with content item instances as a result of including the subset of the one or more modifications in the content item instances, the effect predicted based at least in part on the ranking and the amount of variation in values of the performance metric associated with the plurality of content item instances.