IP Library Granted Patent US 10,445,563
Granted Patent B2
US 10,445,563 · App. 15/718,688 · Granted Oct 15, 2019

Time-in-store estimation using facial recognition

Inventors: Oren Golan (Or-Yehuda, IL); Gadi Zimerman (Hod-Hasharon, IL); Zvi Figov (Modiin, IL); Ofer Waisman (Herzliya Pituach, IL); Elad Falckovitch (Herzliya Pituach, IL); Doron Yankovitch (Herzliya Pituach, IL)
Assignee: VERINT SYSTEMS LTD.
G06K9/00295G06F16/5838G06F16/5854G06F16/5866G06K9/00261G06K9/00281G06K9/00771G06Q30/0201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,445,563
App. No.
15/718,688
Granted
Oct 15, 2019
Kind
B2
Abstract

A method of monitoring the amount of time spent in a specified area by an individual comprises employing a first camera to automatically create one or more entrance images, each entrance image containing a face of an entering individual that passes a first location, and storing each entrance image in a database along with a corresponding entrance time that the entering individual passed the entrance location. An exit image is compared to the entrance images in the database to identify a matching entrance image containing the same face as the exit image. A stay time is then determined for the exiting individual by determining the difference between the entrance time corresponding to the matching entrance image and the exit time.

Claims (111)

1. A non-transitory computer readable medium having instructions stored thereon for monitoring a frequency an individual visits a specified area, in which the instructions when executed by a computing system cause the computing system to at least:

receive an image of an entrance location of a specified area from a camera;

timestamp the image of the entrance location with an associated entrance time; and

detect one or more faces in the image of the entrance location using a face detection algorithm;

create an entrance image of the one or more faces detected in the image of the entrance location;

store the entrance image and the entrance time together in an image database;

detect a matching image by at least using a facial recognition algorithm to compare the entrance image stored in the image database to previously stored face images stored in the image database, wherein the matching entrance image contains a facial match to one of the previously stored face images;

calculate a frequency at which a matched individual visits the specified area based on a comparison of the entrance time stored in the image database corresponding to the entrance image and a time stored in the database corresponding to the matching image.

2. The non-transitory computer readable medium of claim 1 , wherein the instructions further cause the computing system to at least:

calculate an average frequency at with individuals visit the specified area based on the frequency at which the matched individual visits the specified area.

3. The non-transitory computer readable medium of claim 1 , wherein the instructions further cause the computing system to at least:

receive an image of an exit location from a second camera;

timestamp the image of the exit location with an associated exit time; and

detect one or more faces in the image of the exit location using the face detection algorithm;

create an exit image of the one or more faces detected in the image of the exit location;

store the exit image and the exit time together in an exit image database;

determine that the exit image is an unmatched exit image by at least using a facial recognition algorithm to compare the exit image stored in the exit image database to entrance images stored in an entrance image database, wherein the unmatched exit image does not contain a facial match to the entrance images; and

store the unmatched exit image in the image database.

4. The non-transitory computer readable medium of claim 3 , wherein the instructions further causes the computing system to at least:

store entrance images removed from the entrance image database in the image database.

5. The non-transitory computer readable medium of claim 4 , wherein the instructions further causes the computing system to at least:

identify unmatched entrance images in the entrance image database; and

remove unmatched entrance images from the entrance image database that have entrance times before a predefined time period.

6. The non-transitory computer readable medium of claim 1 , wherein receiving an image of the entrance location includes receiving video images of the entrance location.

7. The non-transitory computer readable medium of claim 6 , wherein the instructions further causes the computing system to at least:

continuously process the video images of the entrance location using the face detection algorithm to detect the one or more faces in the image of the entrance location.

8. A method of monitoring a frequency an individual visits a specified area, the method comprising:

receiving an image of an entrance location of a specified area from a camera;

timestamping the image of the entrance location with an associated entrance time; and

detecting one or more faces in the image of the entrance location using a face detection algorithm;

creating an entrance image of the one or more faces detected in the image of the entrance location;

storing the entrance image and the entrance time together in an image database;

detecting a matching image by at least using a facial recognition algorithm to compare the entrance image stored in the image database to previously stored face images stored in the image database, wherein the matching entrance image contains a facial match to one of the previously stored face images;

calculating a frequency at which a matched individual visits the specified area based on a comparison of the entrance time stored in the image database corresponding to the entrance image and a time stored in the database corresponding to the matching image.

9. The method of claim 8 , further comprising:

calculating an average frequency at with individuals visit the specified area based on the frequency at which the matched individual visits the specified area.

10. The method of claim 8 , further comprising:

receiving an image of an exit location from a second camera;

timestamping the image of the exit location with an associated exit time; and

detecting one or more faces in the image of the exit location using the face detection algorithm;

creating an exit image of the one or more faces detected in the image of the exit location;

storing the exit image and the exit time together in an exit image database;

determining that the exit image is an unmatched exit image by at least using a facial recognition algorithm to compare the exit image stored in the exit image database to entrance images stored in an entrance image database, wherein the unmatched exit image does not contain a facial match to the entrance images; and

store the unmatched exit image in the image database.

11. The method of claim 10 , further comprising:

storing entrance images removed from the entrance image database in the image database.

12. The method of claim 11 , further comprising:

identifying unmatched entrance images in the entrance image database; and

removing unmatched entrance images from the entrance image database that have entrance times before a predefined time period.

13. A non-transitory computer readable medium having instructions stored thereon for monitoring a flow pattern of an individual in a specified area, in which the instructions when executed by a computing system cause the computing system to at least:

receive an image of an entrance location of the specified area from a first camera;

timestamp the image of the entrance location with an associated entrance time;

detect one or more faces in the image of the entrance location using a face detection algorithm;

create an entrance image of the one or more faces detected in the image of the entrance location;

store the entrance image and the entrance time together in an entrance image database;

receive an image of a subsection location of the specified area from a second camera;

timestamp the image of the subsection location with an associated subsection time; and

detect one or more faces in the image of the subsection location using the face detection algorithm;

create a subsection image of the one or more faces detected in the image of the subsection location;

store the subsection image and the subsection time together in a subsection image database;

detect a matching subsection image by at least using a facial recognition algorithm to compare the subsection image stored in the subsection image database to one or more entrance images stored in the entrance image database, wherein the matching subsection image contains a facial match to the entrance image;

determine a flow pattern of where the individual visited during a stay within the specified area based on the matching subsection image.

14. The non-transitory computer readable medium of claim 13 , wherein the instructions further causes the computing system to at least:

receive an image of an exit location from a third camera;

timestamp the image of the exit location with an associated exit time; and

detect one or more faces in the image of the exit location using the face detection algorithm;

create an exit image of the one or more faces detected in the image of the exit location;

store the exit image and the exit time together in an exit image database;

detect a matching exit image by at least using a facial recognition algorithm to compare the exit image stored in the exit image database to one or more subsection images stored in the subsection image database, wherein the matching exit image contains a facial match to the subsection image;

determine how long the individual spent in the subsection location of the specified area based on the exit time and the subsection time.

15. The non-transitory computer readable medium of claim 14 , wherein the instructions further causes the computing system to at least:

determine how long the individual spent in the subsection location of the specified area based on the second subsection time and the subsection time.

16. The non-transitory computer readable medium of claim 13 , wherein the instructions further causes the computing system to at least:

receive an image of a second subsection location of the specified area from a third camera;

timestamp the image of the second subsection location with an associated second subsection time; and

detect one or more faces in the image of the second subsection location using the face detection algorithm;

create a second subsection image of the one or more faces detected in the image of the second subsection location;

store the second subsection image and the second subsection time together in a second subsection image database;

detect a matching second subsection image by at least using a facial recognition algorithm to compare the second subsection image stored in the second subsection image database to one or more subsection images stored in the subsection image database, wherein the matching second subsection image contains a facial match to the subsection image;

determine the flow pattern of where the individual visited during a stay within the specified area based on the matching subsection image and the matching second subsection image.

17. A method of monitoring a flow pattern of an individual in a specified area, the method comprising:

receiving an image of an entrance location of the specified area from a first camera;

timestamping the image of the entrance location with an associated entrance time;

detecting one or more faces in the image of the entrance location using a face detection algorithm;

creating an entrance image of the one or more faces detected in the image of the entrance location;

storing the entrance image and the entrance time together in an entrance image database;

receiving an image of a subsection location of the specified area from a second camera;

timestamping the image of the subsection location with an associated subsection time; and

detecting one or more faces in the image of the subsection location using the face detection algorithm;

creating a subsection image of the one or more faces detected in the image of the subsection location;

storing the subsection image and the subsection time together in a subsection image database;

detecting a matching subsection image by at least using a facial recognition algorithm to compare the subsection image stored in the subsection image database to one or more entrance images stored in the entrance image database, wherein the matching subsection image contains a facial match to the entrance image;

determining a flow pattern of where the individual visited during a stay within the specified area based on the matching subsection image.

18. The method of claim 17 , further comprising:

receiving an image of an exit location from a third camera;

timestamping the image of the exit location with an associated exit time; and

detecting one or more faces in the image of the exit location using the face detection algorithm;

creating an exit image of the one or more faces detected in the image of the exit location;

storing the exit image and the exit time together in an exit image database;

detecting a matching exit image by at least using a facial recognition algorithm to compare the exit image stored in the exit image database to one or more subsection images stored in the subsection image database, wherein the matching exit image contains a facial match to the subsection image;

determining how long the individual spent in the subsection location of the specified area based on the exit time and the subsection time.

19. The method of claim 17 , further comprising:

receiving an image of a second subsection location of the specified area from a third camera;

timestamping the image of the second subsection location with an associated second subsection time; and

detecting one or more faces in the image of the second subsection location using the face detection algorithm;

creating a second subsection image of the one or more faces detected in the image of the second subsection location;

storing the second subsection image and the second subsection time together in a second subsection image database;

detecting a matching second subsection image by at least using a facial recognition algorithm to compare the second subsection image stored in the second subsection image database to one or more subsection images stored in the subsection image database, wherein the matching second subsection image contains a facial match to the subsection image;

determining the flow pattern of where the individual visited during a stay within the specified area based on the matching subsection image and the matching second subsection image.

20. The method of claim 19 , further comprising:

determining how long the individual spent in the subsection location of the specified area based on the second subsection time and the subsection time.

Assignments (3)
CHANGE OF NAME Recorded Apr 20, 2022
From: VERINT SYSTEMS LTD.
To: COGNYTE TECHNOLOGIES ISRAEL LTD
Reel/Frame 059710/0742 →
CHANGE OF NAME Recorded Dec 23, 2021
From: VERINT SYSTEMS LTD.
To: COGNYTE TECHNOLOGIES ISRAEL LTD
Reel/Frame 060751/0532 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 23, 2018
From: FALCKOVITCH, ELAD; GOLAN, OREN; ZIMERMAN, GADI; FIGOR, ZVI; WAISMAN, OFER; YANKOVITCH, DORON
To: VERINT SYSTEMS LTD.
Reel/Frame 045336/0151 →
Continuity (5)
Continuation 15432298 · Feb 14, 2017
Continuation 14973074 · Dec 17, 2015
Continuation 14104099 · Dec 12, 2013
Provisional Application 61736437 · Dec 12, 2012
Related Publication 20180082111A1 · Mar 22, 2018