Granted Patent B2

US 12,315,319 · App. 18/318,226 · Granted May 27, 2025

Inventors: Abhijeet Shenoi (San Francisco, CA); Sagar Kashinath Honnungar (Mountain View, CA)

Assignee: HAKIMO INC.

G07C9/15G06T7/70G06V20/41G06V20/52G06V40/168G06V40/172G07C9/00571G06T2207/10016G06T2207/30201G06T2207/30232

Abstract

Claims (87)

1. An electronic device, comprising:

circuitry configured to:

receive data from an access control system for a physical entry point;

detect, based on the data, an event that indicates a grant to access a physical area via the physical entry point;

receive entry information that indicates a set of authentications provided by the access control system within a defined period from a time of the detection of the event;

receive a video from at least one imaging device;

extract, from a portion of the video, object detection information that indicates a set of entries that corresponds to the defined period;

determine cost information based on a difference between a time of each entry of the set of entries and a time of each authentication of the set of authentications;

execute a matching operation on the cost information;

determine a minimum cost association between each authentication of the set of authentications and a corresponding entry of the set of entries based on the executed matching operation;

determine a correspondence between a subset of the set of entries and the set of authentications based on the minimum cost association;

detect, from the set of entries, one or more entries that are different from the subset as one or more tailgating entries; and

control an output device to render tailgating information based on the one or more tailgating entries.

2. The electronic device according to claim 1 , wherein

the physical entry point is a door or a gate, and

the defined period corresponds to a duration for which the door or the gate stays open from the time of the detection of the event.

3. The electronic device according to claim 1 , wherein each authentication of the set of authentications is one of a radio frequency-based authentication, a fingerprint-based authentication, a face-based authentication, an iris-based authentication, a voice-based authentication, a hand gesture-based authentication, a touch-based authentication, a password-based authentication, or an identifier (ID) based authentication.

4. The electronic device according to claim 1 , wherein the circuitry is further configured to:

receive an input comprising a threshold time difference; and

determine the cost information further based on a comparison of the difference with the threshold time difference.

5. The electronic device according to claim 1 , wherein

the cost information is a cost matrix between the set of authentications and the set of entries, and

the cost matrix includes the difference between the time of each entry of the set of entries and the time of each authentication of the set of authentications.

6. The electronic device according to claim 1 , wherein the matching operation corresponds to a Hungarian matching algorithm.

7. The electronic device according to claim 1 , wherein

the set of authentications corresponds to a set of objects, and

each object of the set of objects is a vehicle in which a person is a rider of the vehicle.

8. The electronic device according to claim 1 , wherein

the set of authentications corresponds to a set of objects, and

each object of the set of objects is a person.

9. The electronic device according to claim 1 , wherein the circuitry is further configured to:

detect, with respect to a location of an input interface of a lock system of the physical entry point, a position of a body portion of a person associated with each entry of the set of entries,

wherein the detection of the position of the body portion is based on the video; and

detect at least one entry of the one or more tailgating entries based on the position of the body portion of the at least one entry.

10. The electronic device according to claim 1 , wherein the circuitry is further configured to:

extract a facial image of a person associated with each entry of the set of entries from at least one frame of the video;

determine an identifier associated with the person based on the set of authentications;

extract face information associated with the person from a database based on the identifier;

compare the facial image with the face information; and

detect at least one entry of the one or more tailgating entries based on the comparison of the facial image with the face information.

11. The electronic device according to claim 1 , wherein the circuitry is further configured to:

retrieve a list of blacklisted objects associated with the physical area;

determine an entry of the one or more tailgating entries further based on an association of the entry with at least one item in the list of blacklisted objects; and

control the access control system to restrict an access for the entry based on the association.

12. The electronic device according to claim 1 , wherein the circuitry is further configured to:

extract visual attributes of a person associated with each entry of the set of entries based on an analysis of at least one frame of the video;

determine a whitelisted entry profile for the physical area; and

disregard an entry of the person as the one or more tailgating entries based on a determination that the visual attributes of the person match attributes included in the whitelisted entry profile.

13. The electronic device according to claim 1 , wherein the circuitry is further configured to:

determine, based on an analysis of at least one frame of the video, one or more unsafe items that a person associated with an entry of the one or more tailgating entries possesses; and

control the output device to render information about the determined one or more unsafe items.

14. A method, comprising:

in an electronic device:

receiving data from an access control system for a physical entry point;

detecting, based on the data, an event that indicates a grant to access a physical area via the physical entry point;

receiving entry information comprising a set of authentications provided by the access control system within a defined period from a time of the detection of the event;

receiving a video from at least one imaging device;

extracting, from a portion of the video, object detection information comprising a set of entries that corresponds to the defined period;

determining cost information based on a difference between a time of each entry of the set of entries and a time of each authentication of the set of authentications;

executing a matching operation on the cost information;

determining a minimum cost association between each authentication of the set of authentications and a corresponding entry of the set of entries based on the executed matching operation;

determining a correspondence between a subset of the set of entries and the set of authentications based on the minimum cost association;

detecting, from the set of entries, one or more entries that are different from the subset as one or more tailgating entries; and

controlling an output device to render tailgating information based on the one or more tailgating entries.

15. The method according to claim 14 , wherein the matching operation corresponds to a Hungarian matching algorithm.

16. The method according to claim 14 , further comprising:

detecting, with respect to a location of an input interface of a lock system of the physical entry point, a position of a body portion of a person associated with each entry of the set of entries,

wherein the detection of the position of the body portion is based on the video, and

detecting at least one entry of the one or more tailgating entries based on the position of the body portion of the at least one entry.

17. The method according to claim 14 , further comprising:

extracting a facial image of a person associated with each entry of the set of entries from at least one frame of the video;

determining an identifier associated with the person based on the set of authentications;

extracting face information associated with the person from a database based on the identifier;

comparing the facial image with the face information; and

detecting at least one entry of the one or more tailgating entries based on the comparison of the facial image with the face information.

18. A non-transitory computer-readable medium having stored thereon, computer-executable instructions that, when executed by an electronic device, causes the electronic device to perform operations, the operations comprising:

receiving data from an access control system for a physical entry point;

detecting, based on the data, an event that indicates a grant to access a physical area via the physical entry point;

receiving entry information comprising a set of authentications provided by the access control system within a defined period from a time of the detection of the event;

receiving a video from at least one imaging device;

extracting, from a portion of the video, object detection information comprising a set of entries that corresponds to the defined period;

determining cost information based on a difference between a time of each entry of the set of entries and a time of each authentication of the set of authentications;

executing a matching operation on the cost information;

determining a minimum cost association between each authentication of the set of authentications and a corresponding entry of the set of entries based on the executed matching operation;

determining a correspondence between a subset of the set of entries and the set of authentications based on the minimum cost association;

detecting, from the set of entries, one or more entries that are different from the subset as one or more tailgating entries; and

controlling an output device to render tailgating information based on the one or more tailgating entries.

Assignments (1)

Continuity (1)

References Cited (18)

Cited By (2)

US 12,682,711 US 12,731,450