System, method, and computer program product for monitoring and responding to the performance of an ad
The disclosure is directed to a system, method, and computer program product for detecting and responding to changes in the performance of an ad. In a first phase, an expected performance value for an ad is calculated. In a second phase, the performance of the ad is monitored in intervals, wherein for each interval, the response rate of the ad during the interval is calculated and compared to the expected performance value, an upper performance limit, and a lower performance limit. The ad is replaced or the expected performance value is recalculated, depending on how the response rate compares to the expected performance value, the upper performance limit, and the lower performance limit.
1. A method, performed by a computer system, for detecting and responding to changes in the performance of an ad while ad impressions are being distributed, the method comprising:
monitoring the performance of an ad and calculating a response rate for the ad during a first initialization phase;
calculating an expected performance value based on the response rate in the first initialization phase, wherein calculating the expected performance value in the first initialization phase comprises:
calculating the response rate for the ad during predefined intervals in the first initialization phase,
calculating upper and lower performance limits for each of the predefined intervals in the first initialization phase,
identifying any predefined interval in which either the response rate exceeds the upper performance limit for the predefined interval or falls below the lower performance limit for the predefined interval,
calculating a total response rate during the first initialization phase, wherein data from identified intervals, if any, are excluded from the calculation, and
setting the expected performance value to the calculated total response rate;
in a second phase, monitoring the performance of the ad in intervals, wherein for each interval, the response rate of the ad during the interval is calculated and compared to the expected performance value and upper performance limit and a lower performance limit in the second phase wherein the upper and lower performance limits in the second phase are a function of the expected performance value calculated in the first phase, and wherein the monitoring of the performance of the ad in the first and second phase takes place in substantially real time while impressions of the ad are being displayed or played in an ad space;
in response to the response rate for the ad either (i) falling below the lower performance limit for a first select number of intervals in the second phase or (ii) falling below the expected performance value for a second select number of intervals in the second phase, replacing the ad with a new ad in substantially real time as the response rate is calculated and compared to the expected performance value and the upper and lower performance limits in the second phase; and
in response to the response rate of the ad (i) rising above the upper performance limit for a third select number of intervals in the second phase or (ii) rising above the expected performance value for a fourth select number of intervals in the second phase, calculating a new expected performance value for the ad in substantially real time as the response rate is calculated and compared to the expected performance value and the upper and lower performance limits in the second phase and using the new expected performance value for the ad in calculating the upper and lower performance limits for subsequent intervals in the second phase.
2. The method of claim 1 , wherein the upper and lower performance limits for each interval in the second phase are also a function of the number of impressions of the ad delivered in the interval.
3. The method of claim 1 , wherein the first and third select numbers are each 3, and the second and fourth select numbers are each 5.
4. The method of claim 1 , wherein the intervals are predefined time intervals.
5. The method of claim 1 , wherein, in response to the ad being replaced with a new ad, monitoring the new ad beginning in the second phase using the expected performance value used for the replaced ad.
6. The method of claim 1 , wherein, in response to the ad being replaced with a new ad, repeating the steps of claim 1 with the new ad.
7. A non-transitory, computer-readable medium comprising a computer program that, when executed by a computer system, enables the computer system to perform the following method for detecting and responding to changes in the performance of an ad, the method comprising:
monitoring the performance of an ad and calculating a response rate for the ad during a first initialization phase;
calculating an expected performance value based on the response rate in the first initialization phase, wherein calculating the expected performance value in the first initialization phase comprises:
calculating the response rate for the ad during predefined intervals in the first initialization phase,
calculating upper and lower performance limits for each of the predefined intervals in the first initialization phase,
identifying any predefined interval in which either the response rate exceeds the upper performance limit for the predefined interval or falls below the lower performance limit for the predefined interval,
calculating a total response rate during the first initialization phase, wherein data from identified intervals, if any, are excluded from the calculation, and
setting the expected performance value to the calculated total response rate;
in a second phase, monitoring the performance of the ad in intervals, wherein for each interval, the response rate of the ad during the interval is calculated and compared to the expected performance value and an upper performance limit and a lower performance limit in the second phase, wherein the upper and lower performance limits in the second phase are a function of the expected performance value calculated in the first phase, and wherein the monitoring of the performance of the ad in the first and second phase takes place in substantially real time while impressions of the ad are being displayed or played in an ad space;
in response to the response rate for the ad either (i) falling below the lower performance limit for a first select number of intervals in the second phase or (ii) falling below the expected performance value for a second select number of intervals in the second phase, replacing the ad with a new ad in substantially real time as the response rate is calculated and compared to the expected performance value and the upper and lower performance limits in the second phase; and
in response to the response rate of the ad (i) rising above the upper performance limit for a third select number of intervals in the second phase or (ii) rising above the expected performance value for a fourth select number of intervals in the second phase, calculating a new expected performance value for the ad in substantially real time as the response rate is calculated and compared to the expected performance value and the upper and lower performance limits in the second phase and using the new expected performance value for the ad in calculating the upper and lower performance limits for subsequent intervals in the second phase.
8. The non-transitory, computer-readable medium of claim 7 , wherein the upper and lower performance limits for each interval in the second phase are also a function of the number of impressions of the ad delivered in the interval.
9. The non-transitory, computer-readable medium of claim 7 , wherein the first and third select numbers are each 3, and the second and fourth select numbers are each 5.
10. The non-transitory, computer-readable medium of claim 7 , wherein the intervals are predefined time intervals.
11. The non-transitory, computer-readable medium of claim 7 , wherein, in response to the ad being replaced with a new ad, monitoring the new ad beginning in the second phase using the expected performance value used for the replaced ad.
12. The non-transitory, computer-readable medium of claim 7 , wherein, in response to the ad being replaced with a new ad, repeating the steps of claim 7 with the new ad.
13. A system for detecting and responding to changes in the performance of an ad, the system comprising:
one or more processors;
one or more memory units coupled to the one or more processors, wherein the one or more memory units store instructions that, when executed by the one or more processors, cause the system to perform the operations of:
monitoring the performance of an ad and calculating a response rate for the ad during a first initialization phase;
calculating an expected performance value based on the response rate in the first initialization phase, wherein calculating the expected performance value in the first initialization phase comprises:
calculating the response rate for the ad during predefined intervals in the first initialization phase,
calculating upper and lower performance limits for each of the predefined intervals in the first initialization phase,
identifying any predefined interval in which either the response rate exceeds the upper performance limit for the predefined interval or falls below the lower performance limit for the predefined interval,
calculating a total response rate during the first initialization phase, wherein data from identified intervals, if any, are excluded from the calculation, and
setting the expected performance value to the calculated total response rate;
in a second phase, monitoring the performance of the ad in intervals, wherein for each interval, the response rate of the ad during the interval is calculated and compared to the expected performance value and an upper performance limit and a lower performance limit in the second phase, wherein the upper and lower performance limits in the second phase are a function of the expected performance value calculated in the first phase, and wherein the monitoring of the performance of the ad in the first and second phase takes place in substantially real time while impressions of the ad are being displayed or played in an ad space;
in response to the response rate for the ad either (i) falling below the lower performance limit for a first select number of intervals in the second phase or (ii) falling below the expected performance value for a second select number of intervals in the second phase, replacing the ad with a new ad in substantially real time as the response rate is calculated and compared to the expected performance value and the upper and lower performance limits in the second phase; and
in response to the response rate of the ad (i) rising above the upper performance limit for a third select number of intervals in the second phase or (ii) rising above the expected performance value for a fourth select number of intervals in the second phase, calculating a new expected performance value for the ad in substantially real time as the response rate is calculated and compared to the expected performance value and the upper and lower performance limits in the second phase and using the new expected performance value for the ad in calculating the upper and lower performance limits for subsequent intervals in the second phase.
14. The system of claim 13 , wherein the upper and lower performance limits for each interval in the second phase are also a function of the number of impressions of the ad delivered in the interval.
15. The system of claim 13 , wherein the first and third select numbers are each 3, and the second and fourth select numbers are each 5.
16. The system of claim 13 , wherein the intervals are predefined time intervals.
17. The system of claim 13 , wherein, in response to the ad being replaced with a new ad, monitoring the new ad beginning in the second phase using the expected performance value used for the replaced ad.
18. The system of claim 13 , wherein, in response to the ad being replaced with a new ad, repeating the steps of claim 13 with the new ad.