Method and system for measuring in-store location effectiveness based on shopper response and behavior analysis
The method and system evaluates the effectiveness of a display location within a store based on a behavioral response analysis of shoppers in the vicinity. The effectiveness of a display location is measured by tracking shoppers in-store, extracting and processing shopper attributes, and extracting metrics based on the processed attributes. The metrics of one location is compared to the metrics of another location to determine overall effectiveness.
1. A method, performed by at least a camera, at least a mobile sensor, and at least a computer executing a set of vision and sensor algorithms and a instructions in tangible memory, for evaluating the effectiveness of a display in a location within a store based on a behavior and response analysis of a plurality of people with respect to the display in the location, comprising the following steps of:
a) deploying a plurality of cameras and mobile signal sensors within a vicinity of a display,
wherein the plurality of cameras and mobile signal sensors capture a plurality of images and mobile device signals of a plurality of people in the vicinity of the display,
b) processing the plurality of images and mobile device signals to convert image and mobile signal data into a set of behavior response data, wherein the set of behavior response data comprises at least a set of shopper attributes and shopper experiences in the store,
wherein the shopper attributes comprise location, velocity, body posture, gaze, demographics, and biometrics, wherein location and velocity are determined using mobile device signal triangulation and trilateration or images, body posture is determined using an aggregation of images from different angles or using a human skeleton model, gaze is determined using detection and localization of a face, pupils, or eyes, and demographics are determined using machine learning classification,
c) performing a sensor fusion of a subset of measurements for each shopper attribute for a person using a sensor fusion algorithm, and estimating each shopper attribute through signal and computer vision processing,
d) constructing a trip vector for each position of a person at a given time instance by aggregating a location of the person, a velocity, a timestamp, a body posture, a set of demographics, a visual attention with an item of interest, a physical interaction with an item of interest, an impression level, and a set of biometrics, further comprising the following steps:
i) detecting the visual attention of the set of shoppers on a display by detecting a gaze fixation and a gaze direction and by localizing an intersection point between the gaze direction and a planogram wherein gaze is determined by detecting the eyes or the pupils of the set of shoppers,
ii) detecting the physical interaction of the set of shoppers with a display by estimating the limb position of the set of shoppers and by localizing an intersection point between the limb position and the planogram if the limb position is inside a shelf or an area of the display,
iii) performing an impression analysis by measuring an impression level through a viewership of the set of people,
iv) constructing a trip vector for each position of a person at a given time instance which comprises the set of shopper attributes and a set of contextual parameters,
wherein the set of shopper attributes comprise location, velocity, body posture, demographics, and biometrics, wherein the set of contextual parameters comprise visual attention, physical interaction, and impression level, and
v) storing the trip vectors into a trip database,
wherein the trip vector is extracted at each given time instance consecutively,
e) building a path-to-purchase descriptor for a set of interest points wherein the path-to-purchase descriptor is constructed by extracting a higher-level description of the shopper experience in the store and incorporating temporal analysis of the set of shopper attributes, the set of contextual parameters, and the environmental parameters, further comprising the following steps:
i) refining the trip vectors by removing noise and by smoothing the parameters of the trip vectors if necessary,
ii) detecting the set of interest points based on a set of pre-defined events comprising visual attention, physical interaction, a significant behavior change, and a significant emotional state change,
wherein the significant behavior change comprises a U-turn, a dwell time, or a significant change in location, velocity, motion pattern, and
wherein the significant emotional state change comprises a change in impression level or biometrical state,
iii) constructing a path-to-purchase descriptor at each interest point, which includes a set of event parameters comprising the type of event and the trip vector at the interest point, wherein constructing the path-to-purchase descriptor uses a real-time feed of the trip vectors or recorded trip vectors in the trip database and
iv) recognizing a set shopping activities over multiple time instances comprising label reading, item comparing, and purchasing—by first performing a temporal analysis of the set of shopper attributes and back-propagating a detected shopping activity label to the extracted path-to-purchase descriptors,
f) processing the path-to-purchase descriptors for the set of interest points of a location and correlated point-of-sale data to obtain a set of retail metrics for the location,
g) maintaining controllable parameters pertaining to a display,
wherein the controllable parameters comprise promotion activity, price, advertising activity, design of the products, and marketing elements in order to measure the impact of changes to the controllable parameters across a set of locations of displays, wherein a rule application module is applied to normalize variance within a set of predefined thresholds,
g) evaluating the effectiveness of the display in the location by comparing the set of retail metrics for a set of people with the set of retail metrics for another set of people wherein the set of retail metrics comprises traffic in the vicinity of the display, dwell time, conversion ratios, and purchase events in the location evaluating the effectiveness of the display in the location by comparing the set of retail metrics for a set of people with the set of retail metrics for another set of people wherein the set of retail metrics comprises traffic in the vicinity of the display, dwell time, conversion ratios, purchase events, and other responses in the location in order to generate the rankings of a set of displays in different locations,
wherein the set of retail metrics include traffic statistics and trends and a conversion rate between different stages in a shopper interaction funnel, wherein the different stages in the shopper interaction funnel comprise visit-to-reach rate, reach-to-expose rate, expose-to-visual engage rate, visual-to-physical engage rate, and engage-to-purchase rate stages,
wherein the location effectiveness evaluation comprises a relative performance comparison of the set of displays in different locations,
h) further evaluating the effectiveness of the display in the location by measuring an attraction level based on a viewing time and an engagement level based on a predetermined behavior, and
i) further evaluating the effectiveness of the display in the location by completing a parallel monadic research evaluation process wherein two different display locations, for a product are evaluated in two individual retail stores during a same time period or by completing a sequential monadic research evaluation process, wherein the same display for a product is placed in two different regions of the store.
2. The method according to claim 1 , wherein the method further comprises a step of estimating the location effectiveness based on the contextual description of the shopper experience and a set of point-of-sale data, further comprising the following steps:
a) processing the contextual description of the shopper experience by employing automatic processing or semi-automatic processing in order to measure the location effectiveness using a set of retail metrics across a set of dimensions,
wherein the set of dimensions comprise product, brand, demographic segment, and time,
b) evaluating the location effectiveness based on the set of retail metrics in order to generate the rankings of a set of displays in different locations,
wherein the set of retail metrics include traffic statistics and trends and a conversion rate between different stages in a shopper interaction funnel wherein the different stages comprise visit, reach, expose, visual engagement, physical engagement, and purchase measurements or exposure, engagement, and purchase measurements
wherein the location effectiveness evaluation comprises a relative performance comparison of the set of displays in different locations.
3. The method according to claim 2 , wherein the method further comprises a step of generating a set of indices based on a set of location-sensitive measurements,
wherein the location-sensitive measurements comprises demographic analysis, wherein the demographic analysis comprises traffic count per demographics, dwell time per demographics, and purchase events per demographics.
4. The method according to claim 2 , wherein the method further comprises a step of comparing the effectiveness of a set of displays in different locations synchronously or comparing the effectiveness of a single display at multiple locations at different times.