IP Library Granted Patent US 10,769,913
Granted Patent B2
US 10,769,913 · App. 13/335,591 · Granted Sep 8, 2020

Cloud-based video surveillance management system

Inventors: Lei Wang (Clovis, CA); Hongwei Zhu (Fresno, CA); Farzin Aghdasi (Clovis, CA); Greg Millar (Coarsegold, CA)
Assignee: PELCO, INC.
G08B13/19671G08B13/19643G08B13/19645G08B13/19656H04N7/181H04N21/44008H04N21/4828H04N21/8133
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,769,913
App. No.
13/335,591
Granted
Sep 8, 2020
Kind
B2
Abstract

Systems and methods are described herein that provide a three-tier intelligent video surveillance management system. An example of a system described herein includes a gateway configured to obtain video content and metadata relating to the video content from a plurality of network devices, a metadata processing module communicatively coupled to the gateway and configured to filter the metadata according to one or more criteria to obtain a filtered set of metadata, a video processing module communicatively coupled to the gateway and the metadata processing module and configured to isolate video portions, of video the content, associated with respective first portions of the filtered set of metadata, and a cloud services interface communicatively coupled to the gateway, the metadata processing module and the video processing module and configured to provide at least some of the filtered set of metadata or the isolated video portions to a cloud computing service.

Claims (55)

1. A method of managing a network-based video surveillance system through a computing system in a gateway between a plurality of network devices and a cloud computing service for reducing the amount of video data received by the cloud computing service, the method comprising:

obtaining video content and metadata relating to the video content from a plurality of network devices including first and second network devices that maintain at least some metadata for overlapping geographic areas;

filtering the metadata according to one or more criteria to obtain a filtered set of metadata, wherein filtering the metadata comprises:

identifying a metadata element obtained from the first network device that corresponds to the area for which the first network device and the second network device maintain the at least some metadata elements;

evaluating quality of the identified metadata element according to the one or more criteria;

classifying at least a portion of the metadata element as noise metadata when the evaluated quality it is below a threshold of the one or more criteria;

determining whether a metadata element corresponding to the metadata element having at least a portion classified as noise has been obtained from the second network device;

identifying the metadata element obtained from the first network device as noise if the metadata element corresponding to the metadata element having at least a portion classified as noise has not been obtained from the second network device; and

excluding from the filtered set any metadata element identified as noise to create a noise filtered set of metadata;

determining video frames that are each associated with the noise filtered set of metadata; and

uploading to a cloud computing service at least one of the noise filtered set of metadata or the video frames exclusively associated with the noise filtered set of metadata.

2. The method of claim 1 wherein the plurality of network devices comprise at least one of cameras or video encoders.

3. The method of claim 1 wherein respective ones of the plurality of network devices are associated with respective ones of a plurality of local networks, and wherein each of the plurality of local networks is associated with distinct geographic locations.

4. The method of claim 1 , wherein the filtering further comprises evaluating the quality of the metadata based on at least one of spatial relationships within video content corresponding to the metadata or temporal relationships within the video content corresponding to the metadata.

5. The method of claim 1 further comprising processing the video content to generate one or more supplementary metadata elements.

6. The method of claim 1 further comprising:

receiving a query of at least one of the metadata or the video content; and

processing the query according to one or more predefined rules.

7. The method of claim 1 , further comprising:

maintaining in the filtered set a metadata element for an object that becomes partially or fully obscured, in accordance with temporal relationships between observed objects and pre-defined for permanence and feature continuity.

8. The method of claim 1 , wherein the excluding excludes from the filtered set of metadata at least one metadata element for at least one object classified as noise in response to the at least one object disappearing from the video content within a threshold amount of time from appearing in the video content.

9. The method of claim 1 , wherein filtering the metadata further includes excluding from the filtered set at least one metadata element from the video data from at least one of the at least two neighboring network devices from the plurality of network devices for at least one object determined to be a duplicate of an object in video data from another of the at least two neighboring network devices.

10. The method of claim 1 , wherein a map representation of an object for either the first portions of the filtered set of metadata or the video frames associated with the respective first portions of the filtered set of metadata is uploaded to the cloud computing service in place of the video content or metadata.

11. The method of claim 10 , further comprising: storing, as a backup at the gateway, at least the video content and metadata associated with the representation to be uploaded.

12. The method of claim 1 , wherein the metadata element corresponds to an object or event detected from respective video content.

13. A network-based video surveillance management system comprising:

a gateway configured to obtain video content and metadata relating to the video content from a plurality of network devices including first and second network devices that maintain at least some metadata for overlapping geographic areas;

a metadata processing module communicatively coupled to the gateway and configured to filter the metadata according to one or more criteria to obtain a filtered set of metadata, wherein the metadata processing module configured to filter the metadata is configured to:

identify a metadata element obtained from the first network device that corresponds to the area for which the first network device and the second network device maintain the at least some metadata elements;

evaluate a quality of the identified metadata according to the one or more criteria;

classify at least a portion of the metadata having as noise metadata when the evaluated quality is below a threshold of the one or more criteria;

determine whether a metadata element corresponding to the metadata element having at least a portion classified as noise has been obtained from the second network device;

identify the metadata element obtained from the first network device as noise if the metadata element corresponding to the metadata element having at least a portion classified as noise has not been obtained from the second network device; and

exclude from the filtered set any metadata element identified as noise to create a noise filtered set of metadata;

a video processing module communicatively coupled to the gateway and the metadata processing module and configured to determine video frames that are each associated with the noise filtered set of metadata; and

a cloud services interface communicatively coupled to the gateway, the metadata processing module and the video processing module, the cloud service interface configured to upload to a cloud computing service at least one of the video content associated with the noise filtered set of metadata or the noise filtered set of metadata.

14. The system of claim 13 wherein the plurality of network devices comprise at least one of cameras or video encoders.

15. The system of claim 13 , wherein the metadata processing module is further configured to evaluate the quality of the metadata based on at least one of spatial relationships within video content corresponding to the metadata or temporal relationships within the video content corresponding to the metadata.

16. The system of claim 13 wherein the video processing module is further configured to generate one or more supplementary metadata elements based on the video content.

17. The system of claim 13 further comprising a rule engine communicatively coupled to the gateway, the metadata processing module and the video processing module and configured to receive a query of at least one of the metadata or the video content and to process the query according to one or more predefined rules.

18. A computer program product residing on a non-transitory processor-executable computer storage medium, the computer program product comprising processor-executable instructions configured to cause a processor of a computing system in a gateway between a plurality of network devices and a cloud computing service to:

obtain video content and metadata relating to the video content from a plurality of network devices including first and second network devices that maintain at least some metadata for overlapping geographic areas;

filter the metadata according to one or more criteria to obtain a filtered set of metadata, wherein the processor-executable instructions configured to cause the processor to filter the metadata comprise instruction configured to cause the processor to:

identify a metadata element obtained from the first network device that corresponds to the area for which the first network device and the second network device maintain the at least some metadata elements;

evaluate a quality of the identified metadata element according to the one or more criteria;

classify at least a portion of the metadata as noise when the evaluated quality is below a threshold of the one or more criteria;

determine whether a metadata element corresponding to the metadata element having at least a portion classified as noise has been obtained from the second network device;

identify the metadata element obtained from the first network device as noise if the metadata element corresponding to the metadata element having at least a portion classified as noise has not been obtained from the second network device; and

exclude from the filtered set any metadata element identified as noise to create a noise filtered set of metadata;

determine video frames that are each associated with the noise filtered set of metadata; and

upload to a cloud computing service the noise filtered set of metadata or the video frames associated with the noise filtered set of metadata.

19. The computer program product of claim 18 wherein the plurality of network devices comprise at least one of cameras or video encoders.

20. The computer program product of claim 18 further comprising processor-executable instructions configured to cause the processor to:

receive a query of at least one of the metadata or the video content; and

process the query according to one or more predefined rules.

Assignments (3)
RELEASE OF SECURITY INTERESTS IN PATENTS Recorded Aug 5, 2020
From: WELLS FARGO BANK, NATIONAL ASSOCIATION, A NATIONAL BANKING ASSOCIATION
To: TRANSOM PELCO ACQUISITION, INC. (FORMERLY ZOOM ACQUISITIONCO, INC.); PELCO, INC.
Reel/Frame 053415/0001 →
SECURITY INTEREST Recorded May 29, 2019
From: ZOOM ACQUISITIONCO, INC.; PELCO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION
Reel/Frame 049314/0016 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 19, 2012
From: WANG, LEI; ZHU, HONGWEI; AGHDASI, FARZIN; MILLAR, GREG
To: PELCO, INC.
Reel/Frame 027560/0769 →