Automated campaign configuration switching framework

1. A method comprising:

receiving, over a network connection and at a computing device, a plurality of user requests, originating from respective client devices, for a web content element;

identifying, by the computing device, configuration alternatives, stored in a configuration data store, for the web content element, wherein each respective configuration alternative in the configuration data store has an initial weight stored in a weight data store;

in response to each of the user requests:

selecting, by the computing device, one of the configuration alternatives based on the initial weights;

configuring, by the computing device, an alternative-configured impression of the requested web content according to the selecting; and

presenting the alternative-configured alternative to the respective user request wherein presenting includes transmitting the web content to a respective originating client device;

updating, by the computing device and in the weight data store, each respective initial weight of the configuration alternatives to a respective second weight based on subsequent requests for the web content element corresponding to the presented alterative-configured impression;

automatically selecting, by the computing device and after the updating, a configuration alternative, the configuration alternative having a second weight stored in the weight data store that is below a threshold; and

based on the automatically selecting, increasing, by the computing device and in the weight data store, the second weight of the selected configuration alternative to a third weight by adding a random amount to the second weight.

2. The method of claim 1 , further comprising:

calculating a success metric value for each configuration alternative based on the subsequent requests from users to whom each corresponding alternative-configured impression is presented; and

wherein the updating is in proportion to the success metric each configuration alternative.

3. The method of claim 2 , wherein the updating is further based on a member of the group comprising: statistical significance intervals of the success metric, a number of trials in which each configuration alternative has been tested, a total number of trials in which all configuration alternatives in a campaign have been tested, a frequency of configuration alternative testing trials, timing history of configuration alternative testing trials, dates configuration alternative test impressions, and a geographical location of users viewing a tested configuration alternative.

4. The method of claim 1 , wherein the automatic selecting comprises selecting a configuration alternative with a statistical confidence interval wider than a threshold confidence interval.

5. The method of claim 1 , wherein the automatic selecting comprises selecting a configuration alternative with a number of impressions during a past period of time below a threshold number of impressions.

6. The method of claim 1 , wherein identifying the configuration alternatives comprises identifying information describing a user and selecting a plurality of configuration alternatives based on the information describing the user.

7. The method of claim 6 , wherein identifying information describing the user comprises identifying a user cohort of which the user is a member.

8. A non-transitory computer readable medium comprising instructions, which when executed by at least one processor, configure the processor to perform operations comprising:

receiving, over a network connection and at a computing device, a plurality of user requests, originating from respective client devices, for a web content element;

identifying, by the computing device, configuration alternatives, stored in a configuration data store, for the web content element, wherein each respective configuration alternative in the configuration data store has an initial weight stored in a weight data store;

in response to each of the user requests:

selecting, by the computing device, one of the configuration alternatives based on the initial weights;

configuring, by the computing device, an alternative-configured impression of the requested web content according to the selecting; and

presenting the alternative-configured alternative to the respective user request, wherein presenting includes transmitting the web content to a respective originating client device;

updating, by the computing device and in the weight data store, each respective initial weight of the configuration alternatives to a respective second weight based on subsequent requests for the web content element corresponding to the presented alterative-configured impression;

automatically selecting, by the computing device and after the updating, a configuration alternative, the configuration alternative having a second weight stored in the weight data store that is below a threshold; and

based on the automatically selecting, increasing, by the computing device and in the weight data store, the second weight of the selected configuration alternative to a third weight by adding a random amount to the second weight.

9. The non-transitory computer-readable medium of claim 8 , the operations further comprising:

calculating a success metric value for each configuration alternative based on the subsequent requests from users to whom each corresponding alternative-configured impression is presented; and

wherein the updating is in proportion to the success metric each configuration alternative.

10. The non-transitory computer-readable medium of claim 9 , wherein the updating is further based on a member of the group comprising: statistical significance intervals of the success metric, a number of trials in which each configuration alternative has been tested, a total number of trials in which all configuration alternatives in a campaign have been tested, a frequency of configuration alternative testing trials, timing history of configuration alternative testing trials, dates configuration alternative test impressions, and a geographical location of users viewing a tested configuration alternative.

11. The non-transitory computer-readable medium of claim 8 , wherein the automatic selecting comprises selecting a configuration alternative with a statistical confidence interval wider than a threshold confidence interval.

12. The non-transitory computer-readable medium of claim 8 , wherein the automatic selecting comprises selecting a configuration alternative with a number of impressions during a past period of time below a threshold number of impressions.

13. The non-transitory computer-readable medium of claim 8 , wherein identifying the configuration alternatives comprises identifying information describing a user and selecting a plurality of configuration alternatives based on the information describing the user.

14. The non-transitory computer-readable medium of claim 13 , wherein identifying information describing the user comprises identifying a user cohort of which the user is a member.

15. A system comprising:

at least one processor;

at least one storage device comprising instructions, which when executed by the at least one processor, configure the at least one processor to perform operations comprising:

receiving, over a network connection and at the system a plurality of user requests, originating from respective client devices, for a web content element;

identifying, by the system, configuration alternatives, stored in a configuration data store, for the web content element, wherein each respective configuration alternative in the configuration data store has an initial weight stored in a weight data store;

in response to each of the user requests:

selecting, by the system, one of the configuration alternatives based on the initial weights;

configuring, by the system, an alternative-configured impression of the requested web content according to the selecting; and

presenting the alternative-configured alternative to the respective user request, wherein presenting includes transmitting the web content to a respective originating client device;

updating, by the system and in the weight data store, each respective initial weight of the configuration alternatives to a respective second weight based on subsequent requests for the web content element corresponding to the presented alterative-configured impression;

automatically selecting, by the system and after the updating, a configuration alternative, the configuration alternative having a second weight stored in the weight data store that is below a threshold; and

based on the automatically selecting, increasing, by the system and in the weight data store, the second weight of the selected configuration alternative to a third weight by adding a random amount to the second weight.

16. The system of claim 15 , the operations further comprising:

calculating a success metric value for each configuration alternative based on the subsequent requests from users to whom each corresponding alternative-configured impression is presented; and

wherein the updating is in proportion to the success metric each configuration alternative.

17. The system of claim 16 , wherein the updating is further based on a member of the group comprising: statistical significance intervals of the success metric, a number of trials in which each configuration alternative has been tested, a total number of trials in which all configuration alternatives in a campaign have been tested, a frequency of configuration alternative testing trials, timing history of configuration alternative testing trials, dates configuration alternative test impressions, and a geographical location of users viewing a tested configuration alternative.

18. The system of claim 15 , wherein the automatic selecting comprises selecting a configuration alternative with a statistical confidence interval wider than a threshold confidence interval.

19. The system of claim 15 , wherein the automatic selecting comprises selecting a configuration alternative with a number of impressions during a past period of time below a threshold number of impressions.

20. The system of claim 15 , wherein identifying the configuration alternatives comprises identifying information describing a user and selecting a plurality of configuration alternatives based on the information describing the user.

21. The system of claim 20 , wherein identifying information describing the user comprises identifying a user cohort of which the user is a member.