Methods and apparatus to identify inconsistencies in audience measurement data
Methods, apparatus, systems, and articles of manufacture to identify inconsistencies in audience measurement data are disclosed. Example apparatus disclosed herein are to compare ones of a first set of cumulative audience metrics with one or more limits based on a second set of event-level audience metrics to detect an inconsistency in at least one of the first set of cumulative audience metrics or the second set of event-level audience metrics. Disclosed example apparatus are further to generate a report of the inconsistency in the at least one of the first set of event-level audience metrics or the second set of cumulative audience metrics.
1. A computing system comprising:
a processor; and
a non-transitory computer-readable storage medium, having stored thereon program instructions that, upon execution by the processor, cause performance of a set of operations comprising:
obtaining, at a server associated with an audience measurement entity, audience measurement data corresponding to a plurality of media devices;
determining, using the audience measurement data, a set of cumulative audience metrics and a set of event-level audience metrics,
wherein the set of event-level audience metrics corresponds to metrics associated with media impressions corresponding respectively to a series of events, and
wherein the set of cumulative audience metrics corresponds to metrics associated with a cumulative number of unique users having accessed media at the same series of events;
comparing ones of the set of cumulative audience metrics with one or more limits based on the set of event-level audience metrics at a corresponding event of the series of events to detect one or more inconsistencies in at least one of the set of cumulative audience metrics or the set of event-level audience metrics;
detecting an inconsistency of the one or more inconsistencies based on a difference between a first cumulative audience metric of the set of cumulative audience metrics and a second cumulative audience metric of the set of cumulative audience metrics being greater than an event-level audience metric of the set of event-level audience metrics associated with a target event of the series of events,
wherein the first cumulative audience metric is associated with the target event, and
wherein the second cumulative audience metric is associated with a preceding event relative to the target event; and
causing, in response to detecting the inconsistency of the one or more inconsistencies, the server of the audience measurement entity to stop processing the audience measurement data thereby reducing computational resources associated with the server processing the audience measurement data.
2. The computing system of claim 1 , wherein the set of operations further comprise detecting another inconsistency of the one or more inconsistencies in response to a third cumulative audience metric of the set of cumulative audience metrics associated with the target event being greater than a largest event-level audience metric of the set of event-level audience metrics.
3. The computing system of claim 1 , wherein the set of operations further comprise detecting another inconsistency of the one or more inconsistencies in response to a third cumulative audience metric of the set of cumulative audience metrics associated with the target event being greater than a limit.
4. The computing system of claim 3 , wherein the limit corresponds to a smaller of (i) a sum of event-level audience metrics associated with events ranging from an initial event to the target event in the set of event-level audience metrics or (ii) a universe estimate of a population.
5. The computing system of claim 4 , wherein the universe estimate of the population is an estimate of a total audience that could be exposed to a particular media.
6. The computing system of claim 1 , wherein the set of operations further comprise detecting another inconsistency of the one or more inconsistencies in response to a third cumulative audience metric of the set of cumulative audience metrics associated with the target event being less than a fourth cumulative audience metric of the set of cumulative audience metrics associated with a preceding event relative to the target event.
7. The computing system of claim 1 , wherein the causing, in response to detecting the inconsistency of the one or more inconsistencies, the server of the audience measurement entity to stop processing the audience measurement data further comprises sending an instruction to an audience data provider to stop collecting the audience measurement data, wherein the audience measurement data is obtained from the audience data provider.
8. The computing system of claim 1 , wherein the set of operations further comprise triggering an alert based on the detecting the inconsistency.
9. The computing system of claim 8 , wherein the triggering the alert comprises transmitting the alert to a mobile device.
10. A non-transitory computer-readable storage medium, having stored thereon program instructions which, when executed, cause a processor to perform a set of operations:
obtaining, at a server associated with an audience measurement entity, audience measurement data corresponding to a plurality of media devices;
determining, using the audience measurement data, a set of cumulative audience metrics and a set of event-level audience metrics,
wherein the set of event-level audience metrics corresponds to metrics associated with media impressions corresponding respectively to a series of events, and
wherein the set of cumulative audience metrics corresponds to metrics associated with a cumulative number of unique users having accessed media at the same series of events;
comparing ones of the set of cumulative audience metrics with one or more limits based on the set of event-level audience metrics at a corresponding event of the series of events to detect one or more inconsistencies in at least one of the set of cumulative audience metrics or the set of event-level audience metrics;
detecting an inconsistency of the one or more inconsistencies based on a difference between a first cumulative audience metric of the set of cumulative audience metrics and a second cumulative audience metric of the set of cumulative audience metrics being greater than an event-level audience metric of the set of event-level audience metrics associated with a target event of the series of events,
wherein the first cumulative audience metric is associated with the target event, and
wherein the second cumulative audience metric is associated with a preceding event relative to the target event;
generating, after the detecting, a report of the inconsistency of the one or more inconsistencies; and
causing, in response to detecting the inconsistency of the one or more inconsistencies, the server of the audience measurement entity to stop processing the audience measurement data thereby reducing computational resources associated with the server processing the audience measurement data.
11. The non-transitory computer-readable storage medium of claim 10 , wherein the set of operations further comprise detecting another inconsistency of the one or more inconsistencies in response to a third cumulative audience metric of the set of cumulative audience metrics associated with the target event being less than a largest event-level audience metric of the set of event-level audience metrics.
12. The non-transitory computer-readable storage medium of claim 10 , wherein the set of operations further comprise detecting another inconsistency of the one or more inconsistencies in response to a third cumulative audience metric of the set of cumulative audience metrics associated with the target event being greater than a limit.
13. The non-transitory computer-readable storage medium of claim 12 , wherein the limit corresponds to a smaller of (i) a sum of event-level audience metrics associated with events ranging from an initial event to the target event in the set of event-level audience metrics or (ii) a universe estimate of a population.
14. The non-transitory computer-readable storage medium of claim 10 , wherein the set of operations further comprise detecting another inconsistency of the one or more inconsistencies in response to a third cumulative audience metric of the set of cumulative audience metrics associated with the target event being greater than a fourth cumulative audience metric of the set of cumulative audience metrics associated with a preceding event relative to the target event.
15. The non-transitory computer-readable storage medium of claim 8 , wherein the set of operations further comprise transmitting, in response to the generating, a message to a computing device, the message including the report of the inconsistency of the one or more inconsistencies.
16. A method comprising:
obtaining, at a server associated with an audience measurement entity, audience measurement data corresponding to a plurality of media devices, wherein the audience measurement data is obtained from an audience data provider;
determining, using the audience measurement data, a set of cumulative audience metrics and a set of event-level audience metrics,
wherein the set of event-level audience metrics corresponds to metrics associated with media impressions corresponding respectively to a series of events, and
wherein the set of cumulative audience metrics corresponds to metrics associated with a cumulative number of unique users having accessed media at the same series of events;
comparing ones of the set of cumulative audience metrics with one or more limits based on the set of event-level audience metrics at a corresponding event of the series of events to detect one or more inconsistencies in at least one of the set of cumulative audience metrics or the set of event-level audience metrics;
detecting an inconsistency of the one or more inconsistencies based on a difference between a first cumulative audience metric of the set of cumulative audience metrics and a second cumulative audience metric of the set of cumulative audience metrics being greater than an event-level audience metric of the set of event-level audience metrics associated with a target event of the series of events,
wherein the first cumulative audience metric is associated with the target event, and
wherein the second cumulative audience metric is associated with a preceding event relative to the target event;
causing, in response to detecting the inconsistency of the one or more inconsistencies, the server of the audience measurement entity to stop processing the audience measurement data thereby reducing computational resources; and
transmitting, in response to detecting the inconsistency of the one or more inconsistencies, an alert to a computing device of the audience data provider to report the inconsistency of the one or more inconsistencies.
17. The method of claim 16 , further comprising detecting another inconsistency in response to a third cumulative audience metrics of the set of cumulative audience metrics associated with the target event being less than a largest event-level audience metric of the set of event-level audience metrics.
18. The method of claim 16 , further comprising detecting another inconsistency of the one or more inconsistencies in response to a third cumulative audience metric of the set of cumulative audience metrics associated with the target event being greater than a limit.
19. The method of claim 18 , wherein the limit corresponds to a smaller of (i) a sum of event-level audience metrics associated with events ranging from an initial event to the target event in the set of event-level audience metrics or (ii) a universe estimate of a population.
20. The method of claim 16 , further comprising detecting another inconsistency of the one or more inconsistencies in response to a third cumulative audience metric of the set of cumulative audience metrics associated with the target event being greater than a fourth cumulative audience metric of the set of cumulative audience metrics associated with a preceding event relative to the target event.