IP Library Granted Patent US 11,138,440
Granted Patent B1
US 11,138,440 · App. 16/234,280 · Granted Oct 5, 2021

Systems and methods for automated video classification

Inventors: Jianyu Wang (Mountain View, CA); Lei Huang (Fremont, CA); Guangshuo Liu (Sunnyvale, CA); Renbin Peng (Fremont, CA); Ziheng Wang (San Jose, CA); Di Liu (Belmont, CA)
Assignee: Facebook, Inc.
G06K9/00758G06F16/71G06F16/738G06F16/7837G06K9/00718G06K9/00744
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,138,440
App. No.
16/234,280
Granted
Oct 5, 2021
Kind
B1
Abstract

Systems, methods, and non-transitory computer-readable media can receive a set of video frames associated with a video. For each video frame of the set of video frames, a plurality of interest points are identified based on an interest point detector. For each video frame of the set of video frames, it is determined whether the video frame depicts the same static image as a next video frame in the set of video frames based on the plurality of interest points identified in each video frame.

Claims (47)

1. A computer-implemented method comprising:

receiving, by a computing system, a set of video frames associated with at least a portion of a video;

identifying, by the computing system, for each video frame of the set of video frames, a plurality of interest points based on an interest point detector;

determining, by the computing system, for each video frame of the set of video frames, that the video frame depicts the same static image as a next video frame in the set of video frames based on the plurality of interest points identified in each video frame;

classifying, by the computing system, the video as a static image video based on a determination that a threshold number of video frames from the set of video frames depict the same static image; and

downranking, by the computing system, the video in a content ranking system based on classification of the video as the static image video.

2. The computer-implemented method of claim 1 , wherein the determining, for each video frame of the set of video frames, whether the video frame depicts the same static image as a next video frame comprises determining whether the next video frame comprises a threshold number of interest points in common with the video frame.

3. The computer-implemented method of claim 2 , wherein the determining, for each video frame of the set of video frames, whether the video frame depicts the same static image as a next video frame further comprises determining whether each interest point in common between the video frame and the next video frame has moved less than a threshold distance from the video frame to the next video frame.

4. The computer-implemented method of claim 3 , wherein the threshold distance is a number of pixels.

5. The computer-implemented method of claim 1 , wherein the video comprises a plurality of video frames, and the set of video frames comprises a sampled subset of the plurality of video frames.

6. The computer-implemented method of claim 1 , wherein the video comprises a static image and one or more semi-transparent moving objects overlaid on the static image.

7. The computer-implemented method of claim 1 , wherein the determining, for each video frame of the set of video frames, whether the video frame depicts the same static image as a next video frame comprises:

determining that a first set of consecutive video frames from the set of video frames depicts a first static image, and

determining that a second set of consecutive video frames from the set of video frames depicts a second static image.

8. The computer-implemented method of claim 7 , further comprising:

identifying the first set of consecutive video frames as a first suspicious segment based on a determination that the first set of consecutive video frames comprises at least a threshold number of consecutive video frames, and

identifying the second set of consecutive video frames as a second suspicious segment based on a determination that the second set of consecutive video frames comprises at least the threshold number of consecutive video frames.

9. The computer-implemented method of claim 8 , further comprising classifying the video as a slideshow video based on a determination that the set of video frames comprises a threshold number of suspicious segments.

10. The computer-implemented method of claim 1 , further comprising:

determining, by the computing system, a subject associated with an interest point of the plurality of interest points; and

assigning, by the computing system, an interest point signature to the interest point.

11. A system comprising:

at least one processor; and

a memory storing instructions that, when executed by the at least one processor, cause the system to perform a method comprising:

receiving a set of video frames associated with at least a portion of a video;

identifying, for each video frame of the set of video frames, a plurality of interest points based on an interest point detector;

determining, for each video frame of the set of video frames, that the video frame depicts the same static image as a next video frame in the set of video frames based on the plurality of interest points identified in each video frame;

classifying the video as a static image video based on a determination that a threshold number of video frames from the set of video frames depict the same static image; and

downranking the video in a content ranking system based on classification of the video as the static image video.

12. The system of claim 11 , wherein the determining, for each video frame of the set of video frames, whether the video frame depicts the same static image as a next video frame comprises determining whether the next video frame comprises a threshold number of interest points in common with the video frame.

13. The system of claim 12 , wherein the determining, for each video frame of the set of video frames, whether the video frame depicts the same static image as a next video frame further comprises determining whether each interest point in common between the video frame and the next video frame has moved less than a threshold distance from the video frame to the next video frame.

14. The system of claim 13 , wherein the threshold distance is a number of pixels.

15. The system of claim 11 , wherein the instructions cause the system to perform the method further comprising:

determining a subject associated with an interest point of the plurality of interest points; and

assigning an interest point signature to the interest point.

16. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:

receiving a set of video frames associated with at least a portion of a video;

identifying, for each video frame of the set of video frames, a plurality of interest points based on an interest point detector;

determining, for each video frame of the set of video frames, that the video frame depicts the same static image as a next video frame in the set of video frames based on the plurality of interest points identified in each video frame;

classifying the video as a static image video based on a determination that a threshold number of video frames from the set of video frames depict the same static image; and

downranking the video in a content ranking system based on classification of the video as the static image video.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the determining, for each video frame of the set of video frames, whether the video frame depicts the same static image as a next video frame comprises determining whether the next video frame comprises a threshold number of interest points in common with the video frame.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the determining, for each video frame of the set of video frames, whether the video frame depicts the same static image as a next video frame further comprises determining whether each interest point in common between the video frame and the next video frame has moved less than a threshold distance from the video frame to the next video frame.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the threshold distance is a number of pixels.

20. The non-transitory computer-readable storage medium of claim 16 , wherein the instructions cause the computing system to perform the method further comprising:

determining a subject associated with an interest point of the plurality of interest points; and

assigning an interest point signature to the interest point.

Assignments (2)
CHANGE OF NAME Recorded Nov 23, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058235/0635 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 29, 2019
From: WANG, JIANYU; HUANG, LEI; LIU, GUANGSHUO; PENG, RENBIN; WANG, ZIHENG; LIU, DI
To: FACEBOOK, INC.
Reel/Frame 049023/0223 →
Cited By (5)
US 1,123,954 US 12,242,714 US 12,347,195 US 12,493,990 US 12,541,378