IP Library Granted Patent US 10,776,672
Granted Patent B2
US 10,776,672 · App. 16/265,731 · Granted Sep 15, 2020

Sensor fusion for monitoring an object-of-interest in a region

Inventors: Moussa Doumbouya (Melrose, MA); Yanyan Hu (Medford, MA); Kevin Piette (Carlisle, MA); Pietro Russo (Melrose, MA); Mahesh Saptharishi (Boston, MA); Bo Yang Yu (Winchester, MA)
Assignee: Avigilon Corporation
G06K9/6289A61B5/0077A61B5/015A61B5/0816A61B5/746G01S13/04G06K9/00771G06T7/20G06T7/292G06T7/593G06T7/70G08B21/182H04N5/2253H04N13/204A61B2576/00G06K2209/27G06T2207/10012G06T2207/10021G06T2207/10044G06T2207/10048G06T2207/20084G06T2207/20212G06T2207/30196G06T2207/30232G06T2207/30242
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,776,672
App. No.
16/265,731
Granted
Sep 15, 2020
Kind
B2
Abstract

Methods, systems, and techniques for monitoring an object-of-interest within a region involve receiving at least data from two sources monitoring a region and correlating that data to determine that an object-of-interest depicted or represented in data from one of the sources is the same object-of-interest depicted or represented in data from the other source. Metadata identifying that the object-of-interest from the two sources is the same object-of-interest is then stored for later use in, for example, object tracking.

Claims (56)

1. A method comprising:

receiving image data depicting an object-of-interest within a region and non-image data representing the object-of-interest within the region;

determining that the object-of-interest depicted in the image data is the object-of-interest represented in the non-image data by correlating the image and non-image data;

storing metadata identifying the object-of-interest depicted in the image data and the object-of-interest represented in the non-image data as the same object-of-interest;

determining, using the non-image data, biometric metadata associated with the object-of-interest, the determining the biometric metadata including determining a rate-of-respiration of the object-of-interest;

displaying, on a display, the biometric metadata, the image data depicting the object-of-interest that is associated with the biometric metadata, and an indication that the biometric metadata and the image data depicting the object-of-interest that is associated with the biometric metadata are associated;

determining whether the rate-of-respiration is below a minimum threshold; and

when the rate-of-respiration is below the minimum threshold, triggering a rate-of-respiration alarm, and

wherein the object-of-interest is a person, and the non-image data includes position metadata collected from a radar sensor.

2. The method as claimed in claim 1 wherein the image data and the non-image data collectively represent the person at two locations at two different times, and further comprising displaying, on the display, a tracking indicator indicating that the person has moved between the two locations.

3. The method as claimed in claim 2 wherein the tracking indicator comprises a path indicator showing a path the person has traveled between the two locations.

4. The method as claimed in claim 1 wherein an image sensor that produces the image data is selected from the group consisting of a visible light sensor, a ultraviolet light sensor, an infrared light sensor, a near infrared light sensor, a short-wavelength infrared light sensor, a mid-wavelength infrared light sensor, and a long-wavelength infrared light sensor.

5. The method as claimed in claim 1 wherein the image data is of a field-of-view, the non-image data is of a scanning region, and the field-of-view and the scanning region do not overlap.

6. The method as claimed in claim 1 wherein the image data is of a field-of-view, the non-image data is of a scanning region, and the field-of-view and the scanning region overlap.

7. The method as claimed in claim 1 wherein the image and the non-image data are substantially spatially correlated, and correlating the image and the non-image data is performed using a convolutional neural network.

8. The method as claimed in claim 1 further comprising, prior to determining that the object-of-interest depicted in the image data is the object-of-interest represented in the non-image data by correlating the video and non-image data, receiving a signal to commence a search for the object-of-interest.

9. A method comprising:

receiving image data depicting an object-of-interest within a region and non-image data representing the object-of-interest within the region;

determining that the object-of-interest depicted in the image data is the object-of-interest represented in the non-image data by correlating the image and non-image data;

storing metadata identifying the object-of-interest depicted in the image data and the object-of-interest represented in the non-image data as the same object-of-interest;

determining biometric metadata associated with the object-of-interest, the determining the biometric metadata including determining a temperature of the object-of-interest;

displaying, on a display, the biometric metadata, the image data depicting the object-of-interest that is associated with the biometric metadata, and an indication that the biometric metadata and the image data depicting the object-of-interest that is associated with the biometric metadata are associated;

determining whether the temperature is within a temperature range; and

when the temperature is outside of the temperature range, triggering a temperature alarm, and

wherein the object-of-interest is a person, and the image data includes thermal data.

10. The method as claimed in claim 9 wherein the image data is generated using a camera, the non-image data is generated using a non-image capture device, and the camera comprises a hub for the non-image capture device.

11. The method as claimed in claim 10 further comprising transmitting power to the non-image capture device via the hub.

12. The method as claimed in claim 9 wherein the non-image data is data contained in radiofrequency radiation.

13. The method as claimed in claim 12 wherein the radiofrequency radiation includes Bluetooth™ signals.

14. A method comprising:

receiving image data depicting an object-of-interest within a region and non-image data representing the object-of-interest within the region;

determining that the object-of-interest depicted in the image data is the object-of-interest represented in the non-image data by correlating the image and non-image data;

storing metadata identifying the object-of-interest depicted in the image data and the object-of-interest represented in the non-image data as the same object-of-interest, and

wherein the non-image data is generated using a radar sensor mounted to detect a crossing past a threshold distance from a doorway of a room, and the method further comprising:

using the non-image data from the radar sensor to determine when the object-of-interest has entered the room beyond the threshold distance; and

when the object-of-interest has entered the room beyond the threshold distance, incrementing a count of a number of persons in the room.

15. The method as claimed in claim 14 further comprising:

using the non-image data from the radar sensor to determine when the object-of-interest has transitioned from, relative to the doorway, being outside of the threshold distance to within the threshold distance; and

when the object-of-interest has transitioned, decrementing the count of the number of persons in the room.

16. A system comprising:

an image capture device;

a non-image capture device;

at least one processor communicatively coupled to the image and non-image capture devices; and

a memory communicatively coupled to the at least one processor and having stored thereon computer program code that is executable by the at least one processor, wherein the computer program code, when executed by the at least one processor, causes the at least one processor to perform a method comprising:

receiving image data depicting an object-of-interest within a region and non-image data representing the object-of-interest within the region;

determining that the object-of-interest depicted in the image data is the object-of-interest represented in the non-image data by correlating the image and non-image data;

storing metadata identifying the object-of-interest depicted in the image data and the object-of-interest represented in the non-image data as the same object-of-interest;

determining biometric metadata associated with the object-of-interest, the determining the biometric metadata including determining a temperature of the object-of-interest;

displaying, on a display, the biometric metadata, the image data depicting the object-of-interest that is associated with the biometric metadata, and an indication that the biometric metadata and the image data depicting the object-of-interest that is associated with the biometric metadata are associated;

determining whether the temperature is within a temperature range; and

when the temperature is outside of the temperature range, triggering a temperature alarm, and

wherein the object-of-interest is a person, and the image data includes thermal data.

17. The system as claimed in claim 16 wherein the image data is generated using a camera, the non-image data is generated using a non-image capture device, and the camera comprises a hub for the non-image capture device.

18. The system as claimed in claim 17 further comprising transmitting power to the non-image capture device via the hub.

19. The system as claimed in claim 16 wherein the non-image data is data contained in radiofrequency radiation.

20. The system as claimed in claim 19 wherein the radiofrequency radiation includes Bluetooth™ signals.

Assignments (2)
NUNC PRO TUNC ASSIGNMENT Recorded Aug 31, 2022
From: AVIGILON CORPORATION
To: MOTOROLA SOLUTIONS, INC.
Reel/Frame 061361/0905 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 16, 2019
From: DOUMBOUYA, MOUSSA; PIETTE, KEVIN; RUSSO, PIETRO; SAPTHARISHI, MAHESH; YU, BO YANG; HU, YANYAN
To: AVIGILON CORPORATION
Reel/Frame 049199/0173 →
Continuity (2)
Provisional Application 62662639 · Apr 25, 2018
Related Publication 20190332901A1 · Oct 31, 2019
Cited By (1)
US 12,373,970