IP Library Granted Patent US 11,763,662
Granted Patent B2
US 11,763,662 · App. 17/210,276 · Granted Sep 19, 2023

Systems and methods of enforcing dynamic thresholds of social distancing rules

Inventors: Terence Neill (Lisburn, GB); Alexis B. Ehrlich (Boca Raton, FL)
Assignee: JOHNSON CONTROLS TYCO IP HOLDINGS LLP
G08B21/245G06V10/40G06V20/52G06V40/171G06V40/172
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,763,662
App. No.
17/210,276
Granted
Sep 19, 2023
Kind
B2
Abstract

Example aspects include a method, apparatus and computer-readable medium of enforcing dynamic social distancing rules, comprising monitoring video frames of one or more video feeds, classifying a plurality of persons in the video frames, calculating respective distances between each person of the plurality of persons and remaining persons of the plurality of persons, detecting mask usage of the plurality of persons, measuring a dwell time of the plurality of persons, determining whether the plurality of persons are in violation of the dynamic social distancing rules, and generating an alert. The mask usage indicating whether each person of the plurality of persons is wearing a mask. The dwell time indicating a duration of time that each person of the plurality of persons has been within a proximity threshold to the remaining persons of the plurality of persons.

Claims (90)

1. A method of enforcing dynamic social distancing rules, comprising:

monitoring video frames of one or more video feeds;

classifying a plurality of persons in the video frames;

calculating respective distances between each person of the plurality of persons and remaining persons of the plurality of persons;

detecting mask usage of the plurality of persons, the mask usage indicating whether each person of the plurality of persons is wearing a mask;

selecting a distance threshold based on the mask usage of the plurality of persons;

measuring a dwell time of the plurality of persons, the dwell time indicating a duration of time that each person of the plurality of persons has been within a proximity threshold to the remaining persons of the plurality of persons;

determining whether the plurality of persons are in violation of the dynamic social distancing rules based at least on the respective distances, the distance threshold, the mask usage, and the dwell time; and

generating an alert in response to determining that the plurality of persons are in violation of the dynamic social distancing rules.

2. The method of claim 1 , further comprising:

obtaining identification information of a person of interest; and

identifying a first person of the plurality of persons as the person of interest,

wherein determining whether the plurality of persons are in violation of the dynamic social distancing rules comprises:

selecting a dwell time threshold based at least on whether the person of interest is wearing the mask; and

determining whether the dwell time of the plurality of persons exceeds the dwell time threshold.

3. The method of claim 1 , further comprising:

determining that the dwell time of the plurality of persons exceeds a preliminary dwell time threshold; and

generating a preliminary alert indicating that the plurality of persons have exceeded the preliminary dwell time threshold.

4. The method of claim 1 , wherein determining whether the plurality of persons are in violation of the dynamic social distancing rules comprises:

selecting a dwell time threshold based at least on the mask usage of the plurality of persons; and

determining whether the dwell time of the plurality of persons exceeds the dwell time threshold.

5. The method of claim 1 , wherein generating the alert comprises providing a video clip of the video frames comprising the plurality of persons that are in violation of the dynamic social distancing rules.

6. The method of claim 1 , wherein calculating the respective distances comprises:

determining, based at least on a first size of a first bounding box around a first person of the plurality of persons, a first depth component of a first coordinate set of the first person;

determining, based at least on a second size of a second bounding box around a second person of the plurality of persons, a second depth component of a second coordinate set of the second person; and

calculating a physical distance between the first person and the second person based at least on the first depth component and the second depth component.

7. The method of claim 1 , wherein detecting the mask usage of the plurality of persons comprises:

providing the video frames to a facial feature detection algorithm, the facial feature detection algorithm being configured to extract respective facial features of the plurality of persons; and

determining whether each person of the plurality of persons is wearing the mask based at least on the respective facial features of that person.

8. An apparatus for enforcing dynamic social distancing rules, comprising:

a non-transitory memory storing computer-executable instructions; and

a processor communicatively coupled with the non-transitory memory and configured to execute the computer-executable instructions to:

monitor video frames of one or more video feeds;

classify a plurality of persons in the video frames;

calculate respective distances between each person of the plurality of persons and remaining persons of the plurality of persons;

detect mask usage of the plurality of persons, the mask usage indicating whether each person of the plurality of persons is wearing a mask;

select a distance threshold based on the mask usage of the plurality of persons;

measure a dwell time of the plurality of persons, the dwell time indicating a duration of time that each person of the plurality of persons has been within a proximity threshold to the remaining persons of the plurality of persons;

determine whether the plurality of persons are in violation of the dynamic social distancing rules based at least on the respective distances, the distance threshold, the mask usage, and the dwell time; and

generate an alert in response to determining that the plurality of persons are in violation of the dynamic social distancing rules.

9. The apparatus of claim 8 , wherein the processor is configured to execute further computer-executable instructions to:

obtain identification information of a person of interest; and

identify a first person of the plurality of persons as the person of interest,

wherein to determine whether the plurality of persons are in violation of the dynamic social distancing rules comprises to:

select a dwell time threshold based at least on whether the person of interest is wearing the mask; and

determine whether the dwell time of the plurality of persons exceeds the dwell time threshold.

10. The apparatus of claim 8 , wherein the processor is configured to execute further computer-executable instructions to:

determine that the dwell time of the plurality of persons exceeds a preliminary dwell time threshold; and

generate a preliminary alert indicating that the plurality of persons have exceeded the preliminary dwell time threshold.

11. The apparatus of claim 8 , wherein to determine whether the plurality of persons are in violation of the dynamic social distancing rules comprises to:

select a dwell time threshold based at least on the mask usage of the plurality of persons; and

determine whether the dwell time of the plurality of persons exceeds the dwell time threshold.

12. The apparatus of claim 8 , wherein to generate the alert comprises to provide a video clip of the video frames comprising the plurality of persons that are in violation of the dynamic social distancing rules.

13. The apparatus of claim 8 , wherein to calculate the respective distances comprises to:

determine, based at least on a first size of a first bounding box around a first person of the plurality of persons, a first depth component of a first coordinate set of the first person;

determine, based at least on a second size of a second bounding box around a second person of the plurality of persons, a second depth component of a second coordinate set of the second person; and

calculate a physical distance between the first person and the second person based at least on the first depth component and the second depth component.

14. The apparatus of claim 8 , wherein to detect the mask usage of the plurality of persons comprises to:

provide the video frames to a facial feature detection algorithm, the facial feature detection algorithm being configured to extract respective facial features of the plurality of persons; and

determine whether each person of the plurality of persons is wearing the mask based at least on the respective facial features of that person.

15. A non-transitory computer-readable medium storing computer-readable instructions for enforcing dynamic social distancing rules, executable by a processor to:

monitor video frames of one or more video feeds;

classify a plurality of persons in the video frames;

calculate respective distances between each person of the plurality of persons and remaining persons of the plurality of persons;

detect mask usage of the plurality of persons, the mask usage indicating whether each person of the plurality of persons is wearing a mask;

select a distance threshold based on the mask usage of the plurality of persons;

measure a dwell time of the plurality of persons, the dwell time indicating a duration of time that each person of the plurality of persons has been within a proximity threshold to the remaining persons of the plurality of persons;

determine whether the plurality of persons are in violation of the dynamic social distancing rules based at least on the respective distances, the distance threshold, the mask usage, and the dwell time; and

generate an alert in response to determining that the plurality of persons are in violation of the dynamic social distancing rules.

16. The non-transitory computer-readable medium of claim 15 , wherein the non-transitory computer-readable medium stores further computer-readable instructions, executable by the processor to:

obtain identification information of a person of interest; and

identify a first person of the plurality of persons as the person of interest,

wherein to determine whether the plurality of persons are in violation of the dynamic social distancing rules comprises to:

select a dwell time threshold based at least on whether the person of interest is wearing the mask; and

determine whether the dwell time of the plurality of persons exceeds the dwell time threshold.

17. The non-transitory computer-readable medium of claim 15 , wherein the non-transitory computer-readable medium stores further computer-readable instructions, executable by the processor to:

determine that the dwell time of the plurality of persons exceeds a preliminary dwell time threshold; and

generate a preliminary alert indicating that the plurality of persons have exceeded the preliminary dwell time threshold.

18. The non-transitory computer-readable medium of claim 15 , wherein to determine whether the plurality of persons are in violation of the dynamic social distancing rules comprises to:

select a dwell time threshold based at least on the mask usage of the plurality of persons; and

determine whether the dwell time of the plurality of persons exceeds the dwell time threshold.

19. The non-transitory computer-readable medium of claim 15 , wherein to generate the alert comprises to provide a video clip of the video frames comprising the plurality of persons that are in violation of the dynamic social distancing rules.

20. The non-transitory computer-readable medium of claim 15 ,

wherein to calculate the respective distances comprises to:

determine, based at least on a first size of a first bounding box around a first person of the plurality of persons, a first depth component of a first coordinate set of the first person;

determine, based at least on a second size of a second bounding box around a second person of the plurality of persons, a second depth component of a second coordinate set of the second person; and

calculate a physical distance between the first person and the second person based at least on the first depth component and the second depth component; and

wherein to detect the mask usage of the plurality of persons comprises to:

provide the video frames to a facial feature detection algorithm, the facial feature detection algorithm being configured to extract respective facial features of the plurality of persons; and

determine whether each person of the plurality of persons is wearing the mask based at least on the respective facial features of that person.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 23, 2024
From: JOHNSON CONTROLS TYCO IP HOLDINGS LLP
To: TYCO FIRE & SECURITY GMBH
Reel/Frame 068494/0384 →
NUNC PRO TUNC ASSIGNMENT Recorded Feb 4, 2022
From: JOHNSON CONTROLS US HOLDINGS LLC
To: JOHNSON CONTROLS, INC.
Reel/Frame 058955/0394 →
NUNC PRO TUNC ASSIGNMENT Recorded Feb 4, 2022
From: JOHNSON CONTROLS, INC.
To: JOHNSON CONTROLS TYCO IP HOLDINGS LLP
Reel/Frame 058955/0472 →
NUNC PRO TUNC ASSIGNMENT Recorded Feb 4, 2022
From: SENSORMATIC ELECTRONICS, LLC
To: JOHNSON CONTROLS US HOLDINGS LLC
Reel/Frame 058957/0138 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 7, 2021
From: NEILL, TERENCE; EHRLICH, ALEXIS B.
To: SENSORMATIC ELECTRONICS, LLC
Reel/Frame 055849/0309 →