Granted Patent B1

US 10,262,331 · App. 15/011,442 · Granted Apr 16, 2019

Inventors: Rajeev Sharma (State College, PA); Joonhwa Shin (State College, PA); Youngrock R Yoon (Knoxville, TN); Donghun Kim (State College, PA)

Assignee: VideoMining Corporation

G06Q30/0205H04W4/029

Abstract

Claims (96)

1. A method for tracking of shopper behavior across a plurality of locations using at least a processor to perform the steps of:

a. tracking in-store shopper behavior at a plurality of retail locations using a Shopper Behavior Tracker module, wherein the Shopper Behavior Tracker comprises

i. detecting at least one person in at least one of the plurality of retail locations using an At-Door Shopper Detector module, wherein the At-Door Shopper Detector module further comprises extracting a set of features and estimating demographics information for the at least one person detected,

ii. tracking the movements of said at least one person at said at least one of the plurality of retail locations location, forming at least one trajectory using a Multi-modal Shopper Tracker module,

iii. associating the at least one trajectory with shopper segment data and the at least one trajectory with Point-of-Sale (PoS) data to generate shopper profile data using a Multi-modal Shopper Data Associator module, wherein the Multi-modal Shopper Data Associator module comprises either:

1. detecting the completion of at least one mobile trajectory,

2. retrieving a set of shopper profile data from the in-store shopper database, wherein the shopper profile data contains at least one vision trajectory,

3. performing matching between the at least one vision trajectory and the at least one mobile trajectory,

4. fusing vision trajectories that are associated with the same target at a given time frame using measurement fusion,

5. combining the fused vision trajectories with the mobile trajectory to complete missing segments in the vision trajectories,

or:

1. retrieving a set of shopper profile data from the in-store shopper database, wherein the shopper profile data was created at a time matching a time when the tracked shopper entered or exited the store,

2. computing a similarity of visual features and shopper data, wherein the visual features comprise face and body features, and the shopper data comprise estimated demographics data,

3. determining the best match of the shopper profile data with the estimated demographics data, and

4. combining the best-matched data to create a new shopper profile data,

or,

1. retrieving a set of shopper profile data from the in-store shopper database,

2. retrieving a set of PoS data from a PoS database, wherein the PoS data is created at a time matching a time when the tracked shopper exited the store,

3. identifying categories of items in the retrieved PoS data using a pre-learned item-category mapping table,

4. mapping the location of the categories of items in the store using store layout and planogram information,

5. computing a probabilistic or deterministic measure between the shopper profile data and the PoS data based on the likelihood that the PoS data corresponds to the shopper profile data,

6. determining the best match between the shopper profile data and the PoS data, and

7. associating the instances of the best matching shopper profile data with the PoS data,

iv. populating an in-store shopper database with the shopper profile data, and

v. identifying a revisiting or cross-visiting shopper using shopper profile data from the Multi-modal Shopper Data Associator module and populating the cross-channel shopper database with the result, using a Revisiting and Cross-visiting Shopper Identifier module,

b. populating a cross-channel shopper database with a set of shopper profile data obtained from the Shopper Behavior Tracker module,

c. aggregating shopper profile data for each tracked shopper across multiple locations using the cross-channel shopper database, and

d. profiling shopper behavior using a Shopper Behavior Profiler module.

2. The method of claim 1 , wherein the Shopper Behavior Profiler further comprises estimating derivable metrics for characterizing shopper behavior across multiple locations.

3. The method of claim 1 , wherein the Shopper Behavior Profiler further comprises:

a. deriving metrics for each shopper profile in the cross-channel shopper database, wherein the derivable metrics comprise brand loyalty, shopping style, store loyalty, shopper channel distribution, shopper channel preference, household income, household size, healthy food preference, and organic food preference, and

b. updating a behavioral segment data section of the cross-channel shopper database.

4. The method of claim 1 , further comprising using a Revisiting and Cross-visiting Shopper Identifier module for:

a. determining whether a first set of shopper profile data already exists in the cross-channel shopper database,

b. if a match is found,

i. retrieving the existing shopper profile data from the cross-channel shopper database, and

ii. merging the first set of shopper profile data with the existing shopper profile data,

c. if a match is not found,

i. creating a new shopper profile data, and

d. updating the cross-channel shopper database.

5. The method of claim 1 , further comprising using the cross-channel shopper database, the Shopper Behavior Profiler module, and an Anonymous Panel Creator module for creating an anonymous shopper panel for at least one application using a set of application-specific criteria.

6. The method of claim 1 , wherein the Multi-modal Shopper Tracker module further comprises:

a. capturing a set of images using at least a vision sensor,

b. detecting an object of interest in the set of images and determining the time and location of said object,

c. receiving a data stream from at least a mobile signal sensor,

d. sorting received data from mobile signal sensors from the same source device to bundle them into a single set,

e. calculating a location of the source device using a trilateration method, and

f. tracking the source device using shopper profile data.

7. A system for tracking of shopper behavior across a plurality of locations using at least a processor to perform the steps of:

a. tracking in-store shopper behavior at a plurality of retail locations using a Shopper Behavior Tracker module, wherein the Shopper Behavior Tracker comprises

i. detecting at least one person in at least one of the plurality of retail locations using an At-Door Shopper Detector module, wherein the At-Door Shopper Detector module further comprises extracting a set of features and estimating demographics information for the at least one person detected,

ii. tracking the movements of said at least one person at said at least one of the plurality of retail locations location, forming at least one trajectory using a Multi-modal Shopper Tracker module,

iii. associating the at least one trajectory with shopper segment data and the at least one trajectory with Point-of-Sale (PoS) data to generate shopper profile data using a Multi-modal Shopper Data Associator module, wherein the Multi-modal Shopper Data Associator module comprises either:

1. detecting the completion of at least one mobile trajectory,

2. retrieving a set of shopper profile data from the in-store shopper database, wherein the shopper profile data contains at least one vision trajectory,

3. performing matching between the at least one vision trajectory and the at least one mobile trajectory,

4. fusing vision trajectories that are associated with the same target at a given time frame using measurement fusion,

5. combining the fused vision trajectories with the mobile trajectory to complete missing segments in the vision trajectories,

or:

5. retrieving a set of shopper profile data from the in-store shopper database, wherein the shopper profile data was created at a time matching a time when the tracked shopper entered or exited the store,

6. computing a similarity of visual features and shopper data, wherein the visual features comprise face and body features, and the shopper data comprise estimated demographics data,

7. determining the best match of the shopper profile data with the estimated demographics data, and

8. combining the best-matched data to create a new shopper profile data,

or,

8. retrieving a set of shopper profile data from the in-store shopper database,

9. retrieving a set of PoS data from a PoS database, wherein the PoS data is created at a time matching a time when the tracked shopper exited the store,

10. identifying categories of items in the retrieved PoS data using a pre-learned item-category mapping table,

11. mapping the location of the categories of items in the store using store layout and planogram information,

12. computing a probabilistic or deterministic measure between the shopper profile data and the PoS data based on the likelihood that the PoS data corresponds to the shopper profile data,

13. determining the best match between the shopper profile data and the PoS data, and

14. associating the instances of the best matching shopper profile data with the PoS data,

iv. populating an in-store shopper database with the shopper profile data, and

v. identifying a revisiting or cross-visiting shopper using shopper profile data from the Multi-modal Shopper Data Associator module and populating the cross-channel shopper database with the result, using a Revisiting and Cross-visiting Shopper Identifier module,

b. populating a cross-channel shopper database with a set of shopper profile data obtained from the Shopper Behavior Tracker module,

c. aggregating shopper profile data for each tracked shopper across multiple locations using the cross-channel shopper database, and

d. profiling shopper behavior using a Shopper Behavior Profiler module.

8. The system of claim 7 , wherein the Shopper Behavior Profiler further comprises estimating derivable metrics for characterizing shopper behavior across multiple locations.

9. The system of claim 7 , wherein the Shopper Behavior Profiler further comprises:

a. deriving metrics for each shopper profile in the cross-channel shopper database, wherein the derivable metrics comprise brand loyalty, shopping style, store loyalty, shopper channel distribution, shopper channel preference, household income, household size, healthy food preference, and organic food preference, and

b. updating a behavioral segment data section of the cross-channel shopper database.

10. The system of claim 7 , further comprising using a Revisiting and Cross-visiting Shopper Identifier module for:

a. determining whether a first set of shopper profile data already exists in the cross-channel shopper database,

b. if a match is found,

i. retrieving the existing shopper profile data from the cross-channel shopper database, and

ii. merging the first set of shopper profile data with the existing shopper profile data,

c. if a match is not found,

i. creating a new shopper profile data, and

d. updating the cross-channel shopper database.

11. The system of claim 7 , further comprising using the cross-channel shopper database, the Shopper Behavior Profiler module, and an Anonymous Panel Creator module for creating an anonymous shopper panel for at least one application using a set of application-specific criteria.

12. The system of claim 7 , wherein the Multi-modal Shopper Tracker module further comprises:

a. capturing a set of images using at least a vision sensor,

b. detecting an object of interest in the set of images and determining the time and location of said object,

c. receiving a data stream from at least a mobile signal sensor,

d. sorting received data from mobile signal sensors from the same source device to bundle them into a single set,

e. calculating a location of the source device using a trilateration method, and

f. tracking the source device using shopper profile data.

Assignments (15)

Cited By (14)

US 12,190,285 US 12,231,818 US 12,243,256 US 12,288,294 US 12,321,890 US 12,333,739 US 12,373,971 US 12,400,451 US 12,469,295 US 12,591,796 US 12,635,673 US 12,670,740 US 12,700,110 US 12,705,576