System and method for measuring the relative and absolute effects of advertising on behavior based events over time
The systems and techniques described herein measure advertisement effectiveness of behavior-based outcomes (e.g., site visit, number of pages consumed, searches, online and offline transactions). The system implemented an automated model to measure the impact of exposures and impressions on outcomes using uses panel data, cookie-based data, and combinations thereof. The techniques use test and control approach to calculate effectiveness, where the test group are those exposed to a campaign and a control group who is not exposed. For those exposed, a running analysis of impressions (and other variables) in a pre period is used to determine behavior based outcomes over a set time period after that exposure. As a result, the automated model is able to generate metrics that show absolute and relative impacts on future behavior.
1. A system comprising:
at least one client system comprising at least one storage device and at least one processor configured to execute a monitoring application that collects information about user activity on the Internet and configured to store the information as metric data, the metric data comprising time stamps associated with the user activity for a time period; and
a data processing system comprising one or more storage devices and one or more processors configured to:
access, from the at least one client system, the metric data;
create, based on the time stamps, a record for each discrete time unit of the time period;
parse variables from each record, each variable being associated with a metric in each record;
determine types of the parsed variables;
determine, based on the types of the parsed variables, a first regression modeling function to be applied to a first variable of the parsed variables and a second regression modeling function to be applied to a second variable of the parsed variables;
apply the first regressive modeling function to each metric of the first variable and the second regressive modeling function to each metric of the second variable;
determine, based on the application of the first regressive modeling function and the second regressive modeling function, lift for each discrete time unit of the time period as a difference in an expected outcome of those exposed to at least one impression from those not exposed to the at least one impression; and
generate and store a report based on the lift for each discrete time unit of the time period.
2. The system of claim 1 , wherein the one or more processors are configured to apply the first and second regressive modeling functions by applying at least one statistical binomial modeling function.
3. The system of claim 2 , wherein the one or more processors are configured to apply the at least one statistical binomial modeling function by applying at least one logistic modelling function.
4. The system of claim 1 , wherein the one or more processors are configured to apply the first and second regressive modeling functions by applying at least one statistical continuous modeling function.
5. The system of claim 4 , wherein the one or more processors are configured to apply the at least one statistical continuous modeling function by applying at least one Poisson modeling function with over-dispersion metric for zero bounded continuous actions.
6. The system of claim 1 , wherein the parsed variables from each record comprise at least one of gross impression variables, time adjusted variables, future behavior outcome variables, past behavior outcome variables, or behavior variables.
7. The system of claim 1 , wherein the one or more processors are configured to access the metric data for the time period by accessing impression data and behavior data.
8. The system of claim 7 , wherein the one or more processors are configured to create the record for each discrete time unit of the time period by segmenting the impression data by at least one of a publisher, a publisher type, a creative type, a creative placement, a media channel, or a demographic group.
9. The system of claim 7 , wherein the behavior data are at least one of total internet usage, brand searches, site visits, or purchases.
10. The system of claim 1 , wherein the one or more processors are configured to determine the lift for each discrete time unit of the time period by determining a pre-period before a time in the time period and a post-period after the time in the time period.
11. A method comprising:
collecting, using a monitoring application executed by at least one client system, information about user activity on the Internet;
storing the information as metric data, the metric data comprising time stamps associated with the user activity for a time period;
accessing the metric data;
creating, based on the time stamps, a record for each discrete time unit of the time period;
parsing variables from each record, each variable being associated with a metric in each record;
determining types of the parsed variables;
determining, based on the types of the parsed variables, a first regression modeling function to be applied to a first variable of the parsed variables and a second regression modeling function to be applied to a second variable of the parsed variables;
applying the first regressive modeling function to each metric of the first variable and the second regressive modeling function to each metric of the second variable;
determining, based on the application of the first regressive modeling function and the second regressive modeling function, lift for each discrete time unit of the time period as a difference in an expected outcome of those exposed to at least one impression from those not exposed to the at least one impression; and
generating and storing a report based on the lift for each discrete time unit of the time period.
12. The method of claim 11 , wherein accessing the metric data for the time period comprises accessing impression data and behavior data.
13. A non-transitory computer-readable medium comprising instructions executable by at least one processor to perform a method, the method comprising:
collecting, using a monitoring application executed by at least one client system, information about user activity on the Internet;
storing the information as metric data, the metric data comprising time stamps associated with the user activity for a time period;
accessing the metric data;
creating, based on the time stamps, a record for each discrete time unit of the time period;
parsing variables from each record, each variable being associated with a metric in each record;
determining types of the parsed variables;
determining, based on the types of the parsed variables, a first regression modeling function to be applied to a first variable of the parsed variables and a second regression modeling function to be applied to a second variable of the parsed variables;
applying the first regressive modeling function to each metric of the first variable and the second regressive modeling function to each metric of the second variable;
determining, based on the application of the first regressive modeling function and the second regressive modeling function, lift for each discrete time unit of the time period as a difference in an expected outcome of those exposed to at least one impression from those not exposed to the at least one impression; and
generating and storing a report based on the lift for each discrete time unit of the time period.
14. The non-transitory computer-readable medium of claim 13 , wherein the instructions executable by the at least one processor to perform accessing the metric data for the time period comprise instructions executable by the at least one processor to perform accessing impression data and behavior data.
15. The system of claim 1 , wherein the user activity comprises access of one or more web pages.
16. The method of claim 11 , wherein the user activity comprises access of one or more web pages.
17. The non-transitory computer-readable medium of claim 13 , wherein the user activity comprises access of one or more web pages.
18. The system of claim 1 , wherein the monitoring application is configured to collect information regarding HTTP requests and subsequent HTTP responses.
19. The system of claim 1 , wherein the monitoring application is configured to collect the URLs of web pages accessed by the user, times the web pages were accessed by the user, and an identifier associated with the client system device on which the application is executed.