IP Library Granted Patent US 11,983,927
Granted Patent B1
US 11,983,927 · App. 18/182,922 · Granted May 14, 2024

Methods and apparatus to produce anaomalous video highlights

Inventors: John Peruzzi (San Francisco, CA); Song Cao (Foster City, CA); Rishabh Goyal (San Mateo, CA); Yunchao Gong (Los Altos, CA)
Assignee: Verkada Inc.
G06V20/47G06V10/771
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,983,927
App. No.
18/182,922
Granted
May 14, 2024
Kind
B1
Abstract

A method includes receiving, at a processor and from a requestor compute device, a request including a representation of at least one video camera from a set of video cameras. In response to receiving the request, a set of at least one video file is generated by retrieving a set of video clips associated with the at least one video camera and the time range. For each of the at least one video camera, a subset of video clips is selected from the set of video clips for that video camera based on a comparison of anomaly scores for that video camera, and concatenating the subsets of video clips to generate the set of at least one video file. The method also includes sending a signal to cause display of at least one video file from the set of at least one video file via the requestor compute device.

Claims (50)

1. A non-transitory, processor-readable medium storing instructions to cause a processor to:

receive, from a requestor compute device, a request including a representation of at least one video camera from a plurality of video cameras;

in response to receiving the request, generate a set of at least one video file by:

retrieving a set of video clips associated with the at least one video camera from the plurality of video cameras and a time range,

calculating, for each video camera from the plurality of video cameras, a plurality of anomaly scores based on a feature vector from a plurality of feature vectors generated for that video camera and associated with a first time period,

selecting, for each video camera from the at least one video camera, a subset of video clips from the set of video clips for that video camera based on a comparison of anomaly scores from the plurality of anomaly scores for that video camera, and

concatenating the subsets of video clips to generate the set of at least one video file;

send a signal to cause display of at least one video file from the set of at least one video file via the requestor compute device; and

during a second time period subsequent to the first time period, remove or down-weight at least one feature vector from the plurality of feature vectors based on an age of the at least one feature vector.

2. The non-transitory, processor-readable medium of claim 1 , wherein, for each video camera from the plurality of video cameras, the plurality of anomaly scores is calculated using a different anomaly model for that video camera.

3. The non-transitory, processor-readable medium of claim 1 , further storing instructions to cause the processor to calculate, for each video camera from the plurality of video cameras, the plurality of anomaly for that video camera scores using a k-Nearest Neighbors (KNN) machine learning algorithm.

4. The non-transitory, processor-readable medium of claim 1 , wherein the display of the at least one video file from the set of at least one video file includes a playback window and a plurality of thumbnail images.

5. The non-transitory, processor-readable medium of claim 1 , wherein the request further includes a representation of a desired duration for the set of at least one video file.

6. A method comprising:

identifying, for each video camera from a plurality of video cameras, a representation of a set of feature vectors from a plurality of sets of feature vectors and associated with a time period, each feature vector from the plurality of sets of feature vectors including at least one of location data associated with an optical flow or age data associated with the optical flow;

calculating, for each video camera from the plurality of video cameras and based on the associated representation of the set of feature vectors from the plurality of sets of feature vectors for that video camera, at least one anomaly score from a plurality of anomaly scores;

receiving, from a requestor compute device and subsequent to the time period, a request including (1) a representation of at least one video camera from the plurality of video cameras and (2) a representation of a time range within the time period;

in response to receiving the request, generating a set of at least one video file by:

retrieving a set of video clips associated with the at least one video camera and the time range,

selecting, for each video camera from the at least one video camera, a subset of video clips from the set of video clips for that video camera based on a comparison of the anomaly scores from the plurality of anomaly scores for that video camera, and

concatenating the subsets of video clips to generate the set of at least one video file; and

sending a signal to cause display of at least one video file from the set of at least one video file via the requestor compute device.

7. The method of claim 6 , wherein the display of the at least one video file from the set of at least one video file includes a playback window and at least one of a highlight queue or a plurality of thumbnail images.

8. The method of claim 6 , wherein each feature vector from the plurality of sets of feature vectors includes time data, and speed data, and is associated with a video region in which a moving person has been detected.

9. The method of claim 6 , wherein:

each feature vector from the plurality of sets of feature vectors further includes a first coordinate and a second coordinate substantially orthogonal to the first coordinate,

each of the first coordinate and the second coordinate being generated by averaging the optical flow over a set of at least one video frame.

10. The method of claim 6 , wherein each video clip from the set of video clips has an associated duration of at least about 20 seconds.

11. An apparatus, comprising:

a processor; and

a memory operably coupled to the processor, the memory storing instructions to cause the processor to:

identify, for each video camera from a plurality of video cameras, a representation of a set of feature vectors from a plurality of sets of feature vectors and associated with a first time period;

identify, for each set of feature vectors from the plurality of sets of feature vectors, a representation of an associated time within the first time period;

calculate, for each video camera from the plurality of video cameras and based on the representation of the set of feature vectors from the plurality of sets of feature vectors for that video camera, at least one anomaly score from a plurality of anomaly scores;

receive, from a requestor compute device and subsequent to the first time period, a request including (1) a representation of at least one video camera from the plurality of video cameras and (2) a representation of a time range within the first time period;

in response to receiving the request, generate a set of at least one video file by:

retrieving a set of video clips associated with the at least one video camera and the time range,

selecting, for each video camera from the at least one video camera, a subset of video clips from the set of video clips for that video camera based on a comparison of the at least anomaly score from the plurality of anomaly scores for that video camera, and

concatenating the subset of video clips to generate the set of at least one video file;

send a signal to cause display of at least one video file from the set of at least one video file via the requestor compute device; and

update the plurality of sets of feature vectors during a second time period subsequent to the first time period, by removing at least one feature vector from the plurality of sets of feature vectors in response to detecting that a memory size limit of information has been reached.

12. The apparatus of claim 11 , wherein the request further includes a representation of a desired duration for the at least one video file from the set of at least one video file.

13. The apparatus of claim 11 , wherein the memory further stores instructions to cause the processor to update the plurality of sets of feature vectors during a third time period subsequent to the first time period, by one of removing or down-weighting at least one feature vector from the plurality of sets of feature vectors based on an age of the at least one feature vector from the set of feature vectors.

14. The apparatus of claim 11 , wherein each feature vector from the plurality of sets of feature vectors includes location data, time data, and speed data, and is associated with a video region in which a moving person has been detected.

15. The apparatus of claim 11 , wherein:

each feature vector from the plurality of sets of feature vectors includes location data associated with an optical flow, age data associated with the optical flow, a first coordinate, and a second coordinate substantially orthogonal to the first coordinate,

each of the first coordinate and the second coordinate being generated by averaging the optical flow over a set of at least one video frame.

16. The apparatus of claim 11 , wherein the display of the at least one video file from the set of at least one video file includes a playback window and a highlight queue.

17. The apparatus of claim 11 , wherein the instructions to cause the processor to calculate the plurality of anomaly scores include instructions to cause the processor to calculate the plurality of anomaly scores using a k-Nearest Neighbors (KNN) machine learning algorithm.

18. The apparatus of claim 11 , wherein each video clip from the set of video clips has an associated duration of at least about 20 seconds.

Assignments (2)
SECURITY INTEREST Recorded Oct 1, 2024
From: VERKADA INC.
To: FIRST-CITIZENS BANK & TRUST COMPANY, AS AGENT
Reel/Frame 068758/0910 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 27, 2024
From: GOYAL, RISHABH; CAO, SONG; PERUZZI, JOHN; GONG, YUNCHAO
To: VERKADA INC.
Reel/Frame 066916/0851 →
Cited By (1)
US 12,651,456