Method and system for collecting shopper response data tied to marketing and merchandising elements
The present invention is a method and system for capturing a dataset over time that represents the response of shoppers to a set of marketing and merchandising stimuli or “elements”. The response includes exposure metrics, engagement metrics and conversion metrics for different shopper segments. The shopper segments may be defined by demographics such as gender, age and ethnicity or by the type of trip, such as a quick trip or a fill-up trip. The system comprises a plurality of means for capturing images, such as cameras, covering the area of interest in the vicinity of the marketing or merchandising element. The method comprises automated and semi-automated analysis of the video to extract the shopper behavior and demographics data for computing the defined metrics. The captured data can be further combined with information such a promotions and advertisement outside the store to further enhance the applications of the data collected in-store from the invention. The captured shopper data can be used for many applications such as comparison of the effectiveness of different marketing elements or the relative effectiveness of different types of promotions, variation of the relative effectiveness of different marketing elements over time or differences between the relative effectiveness of a marketing element for different demographic segments.
1. A data collection method for collecting in-store responses and datasets of people tied to marketing and merchandising elements, the method comprising the following steps of:
a) receiving, by a computer from a video processing unit comprising at least a camera that is connected to at least a video interface to capture, a set of video images in a retail space,
b) converting the set of video images to audience data by executing a video processing algorithm, wherein each video image comprises image data and audience data that comprises behavior, demographic, and emotion data,
c) analyzing a plurality of input images to estimate shopper interactions, segmentation, impression level, valences and magnitudes of emotion change, and a unified emotional response of the people, wherein the analysis includes executing computer vision algorithms and 3D facial pose estimation algorithms on the input images, wherein a unified emotional response is considered the persuasiveness of a set of marketing and merchandising elements, wherein a persuasiveness is calculated by and equation
persuasiveness
=
k
Total
impressions
∑
n
-
1
k
∂
emotion
n
,
wherein k is a total number of impressions that had an emotional response to the set of marketing or merchandising elements and demotion is the change in response associated with the nth impression,
d) measuring the responses of the people tied to marketing and merchandising elements based on the analysis for shopper interactions, segmentation, impression level, valences of emotion change, and a unified emotional response,
e) creating one or more datasets by accumulating the response measurements,
f) segmenting the datasets according to demographic information of the people,
g) formulating, by the computer, one or more in-store shopper interaction funnel models in relation to the response measurements, wherein the shopper interaction funnel models allow a systematic consideration of conversion rates at multiple stages of shopper interaction along with related response measurements, wherein the shopper interaction funnel models are organized in a data structure that includes a tree structure or a linked list,
h) correlating the response measurements with conversion rates at multiple stages of shopper interaction,
i) linking the multiple in-store shopper interaction funnel models, defining a set of intersecting behaviors and calculating a sum of numbers between two linked shopper interaction funnel models for an analysis of the response measurements wherein a computer vision-based demographic composition measurement technology measures the demographic information, and wherein the method normalizes values of the datasets,
j) measuring the impact of a first set of marketing and merchandising elements through the measurement of changes occurring in the shopper interaction funnel models of a second set of marketing and merchandising elements, wherein the impact is measured through an equation γ=√{square root over (a 2 +β 2 +ηαβ)}, wherein γ is a combined funnel of funnels α and β, wherein η is a coupling factor between the two funnels that determines whether the first and second set of marketing and merchandising elements support, oppose or do not have an effect on each other, and
k) optimizing, by the computer, the funnel model using the in-store marketing and merchandising elements according to a ranking of performances of said in-store marketing and merchandising elements based on the response measurements.
2. The method according to claim 1 , wherein the method further comprises a step of extrapolating the datasets to get a chain-wide datasets for at least a chain, and associating the extrapolated datasets with planogram, layout of stores, total population of store areas, and predefined characteristics of behavior.
3. The method according to claim 1 , wherein the method further comprises a step of breaking down intermediate stages of the shopper interaction funnel models, designing specific measurement metrics for each intermediate stage, and applying measurement metrics for the response measurements in order to understand the impact of marketing, wherein the types of metrics comprise change in category dynamics, category traffic, category penetration, category conversion rates, activity map, shopping time, shopper demographics, primary and secondary destinations, change in shopper behavior, direction of the travel, change in immediate reaction as reach for product, and change in sales.
4. The method according to claim 1 , wherein the method further comprises a step of correlating a plurality of response measurements in a combination of a plurality of shopper interaction funnel models, and calculating an impact that a marketing and merchandising element has on a shopper interaction funnel model of another marketing and merchandising element.
5. The method according to claim 1 , wherein the method further comprises a step of converting the datasets among a plurality of shopper interaction funnel models comprising time share, category share, and dollar share of customers into standardized outputs through a process of normalization and comparing the standardized outputs, wherein different weights are calculated depending on environments of the marketing and merchandising elements where the shopper interaction funnel models are formulated.
6. The method according to claim 1 , wherein the method further comprises a step of measuring the impact of first marketing and merchandising elements through the measurement of changes occurring in the shopper interaction funnel models of second marketing and merchandising elements.
7. An apparatus for collecting in-store responses and datasets of people tied to marketing and merchandising elements, comprising:
a video processing unit comprising at least a camera that is connected to at least a video interface continuously capturing video images in a retail environment,
a memory storing computer executable instructions, and
a computer operatively coupled to the memory that executes the instructions and configured to perform the steps of:
a) receiving, from the video processing unit, a set of video images,
b) converting the set of video images to audience data by executing a video processing algorithm, wherein each video image comprises image data and audience data that comprises behavior, demographic, and emotion data,
c) analyzing a plurality of input images to estimate shopper interactions, segmentation, impression level, valences and magnitudes of emotion change, and a unified emotional response of the people, wherein the analysis includes executing computer vision algorithms and 3D facial pose estimation algorithms on the input images, wherein a unified emotional response is considered the persuasiveness of a set of marketing and merchandising elements, wherein a persuasiveness is calculated by and equation
persuasiveness
=
k
Total
impressions
∑
n
-
1
k
∂
emotion
n
,
wherein k is a total number of impressions that had an emotional response to the set of marketing or merchandising elements and demotion is the change in response associated with the nth impression,
d) measuring the responses of the people tied to marketing and merchandising elements based on the analysis for shopper interactions, segmentation, impression level, valences of emotion change, and a unified emotional response,
e) creating one or more datasets by accumulating the response measurements,
f) segmenting the datasets according to demographic information of the people,
g) formulating one or more in-store shopper interaction funnel models in relation to the response measurements, wherein the shopper interaction funnel models allow a systematic consideration of conversion rates at multiple stages of shopper interaction along with related response measurements, wherein the shopper interaction funnel models are organized in a data structure that includes a tree structure or a linked list,
h) correlating the response measurements with conversion rates at multiple stages of shopper interaction,
i) linking the multiple in-store shopper interaction funnel models, defining a set of intersecting behaviors and calculating a sum of numbers between two linked shopper interaction funnel models for an analysis of the response measurements wherein a computer vision-based demographic composition measurement technology measures the demographic information, and wherein the method normalizes values of the datasets,
j) measuring the impact of a first set of marketing and merchandising elements through the measurement of changes occurring in the shopper interaction funnel models of a second set of marketing and merchandising elements, wherein the impact is measured through an equation γ=√{square root over (a 2 +β 2 +ηαβ)}, wherein γ is a combined funnel of funnels α and β, wherein η is a coupling factor between the two funnels that determines whether the first and second set of marketing and merchandising elements support, oppose or do not have an effect on each other, and
k) optimizing the funnel model using the in-store marketing and merchandising elements according to a ranking of performances of said in-store marketing and merchandising elements based on the response measurements.
8. The apparatus according to claim 7 , wherein the apparatus further comprises instructions in the memory that, when executed, extrapolate the datasets to get a chain-wide datasets for at least a chain, and associating the extrapolated datasets with planogram, layout of stores, total population of store areas, and predefined characteristics of behavior.
9. The apparatus according to claim 7 , wherein the apparatus further comprises instructions in the memory that, when executed, break down intermediate stages of the shopper interaction funnel models, designing specific measurement metrics for each intermediate stage, and applying measurement metrics for the response measurements in order to understand the impact of marketing, wherein the types of metrics comprise change in category dynamics, category traffic, category penetration, category conversion rates, activity map, shopping time, shopper demographics, primary and secondary destinations, change in shopper behavior, direction of the travel, change in immediate reaction as reach for product, and change in sales.
10. The apparatus according to claim 7 , wherein the apparatus further comprises instructions in the memory that, when executed, correlate a plurality of response measurements in a combination of a plurality of shopper interaction funnel models, and calculating an impact that a marketing and merchandising element has on a shopper interaction funnel model of another marketing and merchandising element.
11. The apparatus according to claim 7 , wherein the apparatus further comprises instructions in the memory that, when executed, convert the datasets among a plurality of shopper interaction funnel models comprising time share, category share, and dollar share of customers into standardized outputs through a process of normalization and comparing the standardized outputs, wherein different weights are calculated depending on environments of the marketing and merchandising elements where the shopper interaction funnel models are formulated.
12. The apparatus according to claim 7 , wherein the apparatus further comprises instructions in the memory that, when executed, measure the impact of first marketing and merchandising elements through the measurement of changes occurring in the shopper interaction funnel models of second marketing and merchandising elements.