Deduplication across multiple different data sources to identify common devices

1 . A system comprising:

a processor; and

at least one memory storing instructions that, when executed by the processor, cause the system to perform operations comprising:

receiving, via first network communications with one or more computing devices, a first set of data obtained by first meter devices;

receiving, via second network communications with the one or more computing devices, a second set of data obtained by second meter devices;

determining whether to apply a restriction to the first set of data obtained by the first meter devices and the second set of data obtained by the second meter devices;

based on determining to apply the restriction, applying the restriction to the first set of data to select a first subset of data, and to the second set of data to select a second subset of data, wherein the first subset of data and the second subset of data each represent a smaller number of media presentation devices than each of the first set of data and the second set of data;

processing the first subset of data and the second subset of data to identify a first media presentation device represented by the first subset of data and a second media presentation device represented by the second subset of data as a possible common media presentation device;

calculating at least one of a station duration metric, a time match metric or a station path metric, wherein:

i) the station duration metric is based on a first set of durations of time that the first media presentation device tuned to a first set of stations and a second set of durations of time that the second media presentation device tuned to the first set of stations,

ii) the time match metric is based on a first set of times of day that the first media presentation device tuned to a second set of stations and a second set of times of day that the second media presentation device tuned to the second set of stations, and

iii) the station path metric based on a first sequence of stations tuned to by the first media presentation device and a second sequence of stations tuned to by the second media presentation device;

determining a score based on the at least one of the station duration metric, the time match metric, or the station path metric; and

determining that the first media presentation device and the second media presentation device are a common media presentation device based on the score.

2 . The system of claim 1 , wherein the operations further comprise:

processing the first set of data by removing data corresponding to the common media presentation device to generate a final data set; and

storing, in the at least one memory, the final data set as deduplicated audience measurement data.

3 . The system of claim 1 , wherein the restriction to the first set of data obtained by the first meter devices and the second set of data obtained by the second meter devices is based on respective physical locations of the first meter devices and the second meter devices.

4 . The system of claim 3 , wherein the respective physical locations comprise the same designated market area.

5 . The system of claim 1 , wherein the first set of stations and the second set of stations at least partially overlap.

6 . The system of claim 1 , wherein the operations further comprise:

calculating a first station duration, q i :

q i =log 10 (ith channel duration for device 1);

calculating a second station duration, p i :

p i =log 10 (ith channel duration for device 2)

calculating a normalized distance score by:

d

⁡

(

p

,

q

)

=

d

⁡

(

q

,

p

)

⁢

∑

i

=

1

n

(

q

i

-

p

i

)

2

d

⁡

(

p

,

q

)

N

0.75

where N is a number of stations; and

setting the normalized distance score as the station duration metric.

7 . The system of claim 1 , wherein the operations further comprise:

dividing the first set of times of day and the second set of times of day into time slots;

identifying a first dominant station tuned to by the first media presentation device in a first time slot of the time slots;

identifying a second dominant station tuned to by the second media presentation device in the first time slot;

performing a first comparison of the first dominant station and the second dominant station;

calculating a first penalty based on the first comparison;

identify a third dominant station tuned to by the first media presentation device in a second time slot of the time slots;

identifying a fourth dominant station tuned to by the second media presentation device in the second time slot;

performing a second comparison of the third dominant station and the fourth dominant station;

calculating a second penalty based on the second comparison;

adding the first penalty and the second penalty to compute a difference score; and

setting the difference score as the time match metric.

8 . The system of claim 7 , wherein the operations further comprise:

setting the first penalty at a first value when at least one of: (i) data for the first time slot is missing for at least one of the first media presentation device or the second media presentation device, or (ii) at least one of the first media presentation device or the second media presentation device is powered off for the first time slot; and

setting the first penalty at a second value when the first media presentation device and the second media presentation device are tuned to different stations during the first time slot, the second value greater than the first value.

9 . The system of claim 1 , wherein the operations further comprise:

determining a number of operations needed to transform the second sequence into the first sequence; and

determining the station path metric based on the number of operations.

10 . The system of claim 9 , wherein the operations further comprise:

computing a Levenshtein distance to determine the number of operations.

11 . The system of claim 1 , wherein the operations further comprise:

determining z-score of the station duration metric;

determining z-score of the time match metric;

determining z-score of the station path metric; and

determining the score based on a combination of the z-score of the station duration metric, the z-score of the time match metric; and the z-score of the station path metric.

12 . The system of claim 1 , wherein the operations further comprise:

determining that the first media presentation device and the second media presentation device are a common media presentation device when the score satisfies a threshold.

13 . The system of claim 12 , wherein the threshold is satisfied when four times a standard deviation of the score is less than a mean of the score.

14 . The system of claim 12 , wherein the first set of durations of time are included in first station duration data, the second set of durations of time are included in second station duration data, the first set of times of day are included in first time data, the second set of times of day are included in second time data, and the first set of durations of time, the second set of durations of time, the first set of times of day, the second set of times of day the first sequence of stations, and the second sequence of stations are associated with a first time interval,

wherein the station duration metric is a first station duration metric, the time match metric is a first time match metric, the station path metric is a first station path metric, the score is a first score, and wherein the operations further comprise:

calculating at least one of a second station duration metric based on third station duration data and fourth station duration data associated with a second time interval, a second time match metric based on third time data and fourth time data associated with the second time internal, or a second station path metric based on a third sequence of stations tuned to by the first media presentation device and a fourth sequence of stations tuned to by the second media presentation device, the third sequence and fourth sequence associated with the second time interval;

determining a second score based on the at least one of the second station duration metric, the second time match metric, or second the station path metric; and

determining that the first media presentation device and the second media presentation device are the common media presentation device based on the second score satisfying the threshold.

15 . The system of claim 1 , wherein the operations further comprise:

calculating a time distance metric based on a first set of time events of activity of the first media presentation device and a second set of time events of the second media presentation device; and

determining the score based on the time distance metric.

16 . A non-transitory computer readable storage medium comprising instructions which, when executed, cause a processor to perform a set of operations comprising:

receiving, via first network communications with one or more computing devices, a first set of data obtained by first meter devices;

receiving, via second network communications with the one or more computing devices, a second set of data obtained by second meter devices;

determining whether to apply a restriction to the first set of data obtained by the first meter devices and the second set of data obtained by the second meter devices;

based on determining to apply the restriction, applying the restriction to the first set of data to select a first subset of data, and to the second set of data to select a second subset of data, wherein the first subset of data and the second subset of data each represent a smaller number of media presentation devices than each of the first set of data and the second set of data;

processing the first subset of data and the second subset of data to identify a first media presentation device represented by the first subset of data and a second media presentation device represented by the second subset of data as a possible common media presentation device;

calculating at least one of a station duration metric, a time match metric or a station path metric, wherein:

i) the station duration metric is based on a first set of durations of time that the first media presentation device tuned to a first set of stations and a second set of durations of time that the second media presentation device tuned to the first set of stations,

ii) the time match metric is based on a first set of times of day that the first media presentation device tuned to a second set of stations and a second set of times of day that the second media presentation device tuned to the second set of stations, and

iii) the station path metric based on a first sequence of stations tuned to by the first media presentation device and a second sequence of stations tuned to by the second media presentation device;

determining a score based on the at least one of the station duration metric, the time match metric, or the station path metric; and

determining that the first media presentation device and the second media presentation device are a common media presentation device based on the score.

17 . The non-transitory computer readable storage medium of claim 16 , wherein the operations further comprise:

processing the first set of data by removing data corresponding to the common media presentation device to generate a final data set; and

storing, in a memory, the final data set as deduplicated audience measurement data.

18 . The non-transitory computer readable storage medium of claim 16 , wherein the restriction to the first set of data obtained by the first meter devices and the second set of data obtained by the second meter devices is based on respective physical locations of the first meter devices and the second meter devices.

19 . The non-transitory computer readable storage medium of claim 18 , wherein the respective physical locations comprise the same designated market area.

20 . A method comprising:

receiving, via first network communications with one or more computing devices, a first set of data obtained by first meter devices;

receiving, via second network communications with the one or more computing devices, a second set of data obtained by second meter devices;

determining whether to apply a restriction to the first set of data obtained by the first meter devices and the second set of data obtained by the second meter devices;

based on determining to apply the restriction, applying the restriction to the first set of data to select a first subset of data, and to the second set of data to select a second subset of data, wherein the first subset of data and the second subset of data each represent a smaller number of media presentation devices than each of the first set of data and the second set of data;

processing the first subset of data and the second subset of data to identify a first media presentation device represented by the first subset of data and a second media presentation device represented by the second subset of data as a possible common media presentation device;

calculating at least one of a station duration metric, a time match metric or a station path metric, wherein:

i) the station duration metric is based on a first set of durations of time that the first media presentation device tuned to a first set of stations and a second set of durations of time that the second media presentation device tuned to the first set of stations,

ii) the time match metric is based on a first set of times of day that the first media presentation device tuned to a second set of stations and a second set of times of day that the second media presentation device tuned to the second set of stations, and

iii) the station path metric based on a first sequence of stations tuned to by the first media presentation device and a second sequence of stations tuned to by the second media presentation device;

determining a score based on the at least one of the station duration metric, the time match metric, or the station path metric; and

determining that the first media presentation device and the second media presentation device are a common media presentation device based on the score.