Method and system for media audience measurement by viewership extrapolation based on site, display, and crowd characterization
View Patent ↗The present invention provides a comprehensive method to design an automatic media audience measurement system that can estimate the site-wide audience of a media of interest (e.g., the site-wide viewership of a target display) based on the measurements of a subset of the actual audience sampled from a limited space in the site. This invention enables (1) the placement of sensors in optimal positions for the viewership data measurement and (2) the estimation of the site-wide viewership of the target display by performing the viewership extrapolation based on the sampled viewership data. The viewership extrapolation problem is formulated in a way that the time-varying crowd dynamics around the target display is an important decisive factor as well as the sampled viewership data at a given time in yielding the estimated site-wide viewership. To solve this problem, the system elements that affect the viewership—site, display, crowd, and audience—and their relationships are first identified in terms of the visibility, the viewership relevancy, and the crowd occupancy. The optimal positions of the sensors are determined to cover the maximum area of the viewership with high probabilities. The viewership extrapolation function is then modeled and learned from the sampled viewership data, the site-wide viewership data, and the crowd dynamics measurements while removing the noise in the sampled viewership data using the viewership relevancy of the measurements to the target display.
1. A method for building a media audience measurement system based on a viewership extrapolation of measured media audience in a site, comprising the following steps of:
a) estimating, by a computer, viewership relevancy distribution and viewer occupancy distribution,
b) estimating, by the computer, viewership distribution from both the viewership relevancy distribution and the viewer occupancy distribution,
c) placing a plurality of sensors in optimal positions to measure viewership data in the site with given constraints that include budget and hardware specifications of available sensors,
d) modeling features for capturing viewership and crowd dynamics,
e) modeling functions for extrapolation-function learning,
f) measuring, by the computer, a first set of sampled viewership data, a first set of site-wide viewership data, and a first set of crowd dynamics data,
g) capturing a plurality of images using a plurality of sensors deployed in the site,
h) dividing each image into multiple regions in a grid,
i) extracting optical flows of each image from each region in the grid,
j) constructing a histogram of the optical flows according to their orientations for each region in the grid,
k) concatenating histograms of multiple regions, whereby the crowd dynamics data of an image can be represented by a single feature vector,
l) estimating, by the computer, a first set of true sampled viewership data from the first set of sampled viewership data,
m) learning viewership extrapolation functions based on relationship between the first set of sampled viewership data and the first set of site-wide viewership data while taking into account the first set of crowd dynamics data, and
n) extrapolating, by the computer, a second set of site-wide viewership data using a second set of sampled viewership data and a second set of crowd dynamics data when the media audience measurement system is deployed in the site, wherein the viewership distribution is defined as probability distribution that a viewer is located at a certain position in a site and a viewer is engaged with a display.
2. The method according to claim 1 , wherein the method further comprises a step of estimating the viewership relevancy distribution, further comprising the following steps of:
a) modeling a visibility distribution based on geographic and geometric attributes of each display in the site,
b) estimating visibility distributions of displays in the site, and
c) estimating the viewership relevancy distribution based on the visibility distributions, taking into account a viewership measurement algorithm,
wherein the visibility distribution is a probability distribution that a viewer is engaged with a display, given a viewer at a position.
3. The method according to claim 1 , wherein the method further comprises a step of estimating the viewer occupancy distribution based on a collection of images of viewers in the site, which is taken by using a plurality of sensors with an occupancy measurement algorithm,
wherein the viewer occupancy distribution indicates how likely a viewer for any display is located at a position in the site, and
wherein crowd becomes the viewers at a given time when the crowd is located in the site and engaged with displays.
4. The method according to claim 1 , wherein the method further comprises a step of estimating the first set of true sampled viewership data from the first set of sampled viewership data using the viewership relevancy distribution of a first target display by a viewership noise removal process,
wherein the first set of sampled viewership data is modeled as a subset of the true viewers of the first target display with a noise.
5. The method according to claim 1 , wherein the method further comprises a step of measuring the first set of sampled viewership data, further comprising a series of the following computer vision steps:
a) capturing a plurality of images in the site,
b) extracting features of viewers in the images captured from a plurality of sensors,
c) detecting faces of the viewers,
d) tracking the faces of the viewers,
e) estimating 3D pose of the faces of the viewers, and
f) collecting the first set of sampled viewership data.
6. The method according to claim 1 , wherein the method further comprises a step of estimating the second set of site-wide viewership data based on the viewership extrapolation, further comprising the following steps:
a) measuring the second set of sampled viewership data and the second set of crowd dynamics data,
b) estimating a second set of true sampled viewership data from the second set of sampled viewership data using the viewership relevancy distribution at the site,
c) finding a viewership extrapolation function in a database of the viewership extrapolation functions, and
d) extrapolating the second set of site-wide viewership data.
7. The method according to claim 6 , wherein the method further comprises a step of estimating the second set of true sampled viewership data from the second set of sampled viewership data using the viewership relevancy distribution of a second target display by a viewership noise removal process,
wherein the second set of sampled viewership data is modeled as a subset of the true viewers of the second target display with a noise.
8. The method according to claim 6 , wherein the method further comprises a step of measuring the second set of sampled viewership data, further comprising a series of the following computer vision steps:
a) capturing a plurality of images in the site,
b) extracting features of viewers in the images captured from a plurality of sensors,
c) detecting faces of the viewers,
d) tracking the faces of the viewers,
e) estimating 3D pose of the faces of the viewers, and
f) collecting the second set of sampled viewership data.
9. A media audience measurement system based on a viewership extrapolation of measured media audience in a site, comprising:
a computer that performs the following steps of:
a) estimating, by the computer, viewership relevancy distribution and viewer occupancy distribution,
b) estimating, by the computer, viewership distribution from both the viewership relevancy distribution and the viewer occupancy distribution, and
c) placing a plurality of sensors in optimal positions to measure viewership data in the site with given constraints that include budget and hardware specifications of available sensors,
d) modeling features for capturing viewership and crowd dynamics,
e) modeling functions for extrapolation-function learning,
f) sensors for measuring a first set of sampled viewership data, a first set of site-wide viewership data, and a first set of crowd dynamics data,
g) a plurality of sensors for capturing a plurality of images in the site, and
h) dividing each image into multiple regions in a grid,
i) extracting optical flows of each image from each region in the grid,
j) constructing a histogram of the optical flows according to their orientations for each region in the grid,
k) concatenating histograms of multiple regions, whereby the crowd dynamics data of an image can be represented by a single feature vector,
l) first learning machine for estimating a first set of true sampled viewership data from the first set of sampled viewership data,
m) a second learning machine for learning viewership extrapolation functions based on relationship between the first set of sampled viewership data and the first set of site-wide viewership data while taking into account the first set of crowd dynamics data, and
n) extrapolating, by the computer, a second set of site-wide viewership data using a second set of sampled viewership data and a second set of crowd dynamics data when the media audience measurement system is deployed in the site.
10. The apparatus according to claim 9 , wherein the apparatus further comprises a computer for estimating the viewership relevancy distribution, performing the following steps of:
a) modeling a visibility distribution based on geographic and geometric attributes of each display in the site,
b) estimating visibility distributions of displays in the site, and
c) estimating the viewership relevancy distribution based on the visibility distributions, taking into account a viewership measurement algorithm,
wherein the visibility distribution is a probability distribution that a viewer is engaged with a display, given a viewer at a position.
11. The apparatus according to claim 9 , wherein the apparatus further comprises a computer for estimating the viewer occupancy distribution based on a collection of images of viewers in the site, which is taken by using a plurality of sensors with an occupancy measurement algorithm.
12. The apparatus according to claim 9 , wherein the apparatus further comprises a computer for estimating the first set of true sampled viewership data from the first set of sampled viewership data using the viewership relevancy distribution of a first target display by a viewership noise removal process.
13. The apparatus according to claim 9 , wherein the apparatus further comprises:
a) a plurality of sensors for capturing a plurality of images in the site, and
b) a computer that performs the following computer vision steps of:
extracting features of viewers in images captured from a plurality of sensors,
detecting faces of the viewers,
tracking the faces of the viewers,
estimating 3D pose of the faces of the viewers, and
collecting the first set of sampled viewership data.
14. The apparatus according to claim 9 , wherein the apparatus further comprises a computer for estimating the second set of site-wide viewership data based on the viewership extrapolation, further comprising:
a) sensors for measuring the second set of sampled viewership data and the second set of crowd dynamics data, and
b) a computer that performs the following steps of:
estimating a second set of true sampled viewership data from the second set of sampled viewership data using the viewership relevancy distribution at the site,
finding a viewership extrapolation function in a database of the viewership extrapolation functions, and
extrapolating the second set of site-wide viewership data.
15. The apparatus according to claim 14 , wherein the apparatus further comprises a computer for estimating the second set of true sampled viewership data from the second set of sampled viewership data using the viewership relevancy distribution of a second target display by a viewership noise removal process.
16. The apparatus according to claim 14 , wherein the apparatus further comprises
a) a plurality of sensors for capturing a plurality of images in the site, and
b) a computer that performs the following computer vision steps of:
extracting features of viewers in the images captured from a plurality of sensors,
detecting faces of the viewers,
tracking the faces of the viewers,
estimating 3D pose of the faces of the viewers, and
collecting the second set of sampled viewership data.