IP Library Granted Patent US 10,373,458
Granted Patent B2
US 10,373,458 · App. 15/492,011 · Granted Aug 6, 2019

Automatic threat detection based on video frame delta information in compressed video streams

Inventors: David Lee Selinger (Pleasanton, CA); Ching-Wa Yip (Hong Kong, CH); Chaoying Chen (Taiwan, CH)
G08B13/19602B60R25/1012G08B13/196G08B13/19667H04N7/18H04N7/181H04N21/21805G06K9/00771
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,373,458
App. No.
15/492,011
Granted
Aug 6, 2019
Kind
B2
Abstract

Apparatus and associated methods relate to identifying objects of interest and detecting motion to automatically detect a security threat as a function of video frame delta information received from a video encoder. In an illustrative example, the video encoder may be an H.264 encoder onboard a video camera. A cloud server may receive the video frame delta information in a compressed video stream from the camera. Threats may be detected by the cloud server processing the video frame delta information in the compressed video stream, without decompression, to identify objects and detect motion. The cloud server may employ artificial intelligence techniques to enhance threat detection by the cloud server. Various examples may advantageously provide increased capacity of a computer tasked with detecting security breaches, due to the significant reduction in the amount of data to be processed, relative to threat detection based on processing uncompressed video streams.

Claims (25)

1. A system to detect an event in compressed video, comprising:

a plurality of surveillance cameras, each of the plurality of surveillance cameras comprising:

a video encoder, configured to compress video and emit frames of a compressed video stream comprising video frame delta information, wherein said video frame delta information excludes redundant data in the form of pixels in each frame that do not change relative to a reference frame and identifies pixels that have changed relative to a prior frame;

a processor, communicatively coupled to the video encoder;

a memory that is not a transitory propagating signal, the memory connected to the processor and encoding computer readable instructions, including processor executable program instructions, the computer readable instructions accessible to the processor, wherein the processor executable program instructions, when executed by the processor, cause the processor to perform operations comprising:

receiving, from the video encoder, a series of video frames comprising video frame delta information; and

identifying objects of interest and detecting motion to detect a security threat as a function of the video frame delta information; and

a cloud server communicatively coupled to the plurality of surveillance cameras, the cloud server comprising:

a second processor, operably coupled to a memory that is not a transitory propagating signal, the memory connected to the second processor and comprising second computer readable instructions, including second processor executable program instructions, the second computer readable instructions accessible to the second processor, wherein the second processor executable program instructions, when executed by the second processor, cause the processor to perform operations comprising:

sending, to at least one surveillance camera, an electronic message comprising a pattern representative of an event to be detected; and

receiving, from the at least one surveillance camera, an electronic message comprising an indication of a detected event based at least in part on said pattern representative of said event,

wherein the cloud server memory further comprises a predictive model adapted to estimate an event probability determined as a function of video frame delta information; the operations performed by the cloud server processor further comprise sending the predictive model to the surveillance camera; and the operations performed by the surveillance camera processor further comprise: receiving the predictive model from the cloud server; estimating an event probability determined as a function of video frame delta information applied as input data to the predictive model; and upon a determination the estimate of the event probability satisfies a predetermined criterion, sending the estimate of the event probability to the cloud server.

2. The system of claim 1 , in which the operations performed by the surveillance camera processor to identify objects of interest and detect motion as a function of the video frame delta information further comprise a technique selected from the group consisting of optical flow, edge detection, boundary detection, feature extraction, speeded up robust features, scale-invariant feature transform, and template matching.

3. A system to detect an event in compressed video, comprising:

a plurality of surveillance cameras, each of the plurality of surveillance cameras comprising:

a video encoder, configured to compress video and emit frames of a compressed video stream comprising video frame delta information;

a processor, communicatively coupled to the video encoder;

a memory that is not a transitory propagating signal, the memory connected to the processor and encoding computer readable instructions, including processor executable program instructions, the computer readable instructions accessible to the processor, wherein the processor executable program instructions, when executed by the processor, cause the processor to perform operations comprising:

receiving, from the video encoder, a series of video frames comprising video frame delta information; and

identifying objects of interest and detecting motion to detect a security threat as a function of the video frame delta information; and

a cloud server communicatively coupled to the plurality of surveillance cameras, the cloud server comprising:

a second processor, operably coupled to a memory that is not a transitory propagating signal, the memory connected to the second processor and comprising second computer readable instructions, including second processor executable program instructions, the second computer readable instructions accessible to the second processor, wherein the second processor executable program instructions, when executed by the second processor, cause the processor to perform operations comprising:

sending, to at least one surveillance camera, an electronic message comprising a pattern representative of an event to be detected; and

receiving, from the at least one surveillance camera, an electronic message comprising an indication of a detected event based at least in part on said pattern representative of said event,

wherein the cloud server memory comprises a predictive model adapted to estimate an event probability determined as a function of video frame delta information; the operations performed by the cloud server processor further comprise sending the predictive model to the surveillance camera; and the operations performed by the surveillance camera processor further comprise: receiving the predictive model from the cloud server; estimating an event probability determined as a function of video frame delta information applied as input data to the predictive model; and upon a determination the estimate of the event probability satisfies a predetermined criterion, sending the estimate of the event probability to the cloud server.

Assignments (2)
SECURITY INTEREST Recorded Sep 10, 2025
From: DEEP SENTINEL CORP.
To: TRIPLEPOINT CAPITAL LLC, AS COLLATERAL AGENT
Reel/Frame 072218/0623 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 7, 2018
From: SELINGER, DAVID; CHEN, CHAOYING; YIP, CHING-WA
To: DEEP SENTINEL CORP.
Reel/Frame 045736/0052 →
Continuity (1)
Related Publication 20180308330A1 · Oct 25, 2018
Cited By (5)
US 12,218,697 US 12,236,762 US 12,417,136 US 12,549,687 US 12,719,496