IP Library › Granted Patent US 11,581,019
Granted Patent B2
US 11,581,019 · App. 17/479,096 · Granted Feb 14, 2023

Automated video editing

Inventors: Kenneth Jay Spreitzer (San Francisco, CA); Haoyun Wu (Rancho Palos Verdes, CA)
Assignee: Snap Inc.
G11B27/031G06T7/20G06V20/40G06T2207/10016
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,581,019
App. No.
17/479,096
Granted
Feb 14, 2023
Kind
B2
Abstract

A method of generating a modified video file using one or more processors is disclosed. The method comprises detecting objects that are represented in an original video file using computer vision object-detection techniques, determining object motion characteristics for the detected objects, based on a specific object motion characteristic for a specific detected object meeting certain requirements, selecting a corresponding audio or visual effect, and applying the corresponding visual effect to the original video file to create the modified video file.

Claims (46)

1. A method, executed by one or more processors, for generation of a modified video file for presentation on a device including a display, comprising:

initiating scanning of an original video file;

detecting objects represented in the original video file using computer vision object-detection techniques;

determining object motion characteristics for the detected objects;

based on the object motion characteristics not meeting a suitability criterion, terminating the method;

based on the object motion characteristics meeting a suitability criterion, performing the following steps;

based on a specific object motion characteristic for a specific detected object meeting a predetermined requirement, selecting a relevant audio or visual effect;

applying the relevant audio or visual effect to the original video file to create the modified video file; and

providing a notification of the existence of the modified video file on the display of the device.

2. The method of claim 1 wherein the predetermined requirement for the specific object motion characteristic is a change in direction of the specific detected object.

3. The method of claim 2 wherein the relevant audio or visual effect is a sound effect applied to correspond to the change in direction of the specific detected object.

4. The method of claim 1 wherein the predetermined requirement for the specific object motion characteristic is repeating motion of the specific detected object.

5. The method of claim 4 wherein the relevant audio or visual effect is a sound effect that is applied repetitively at a frequency of motion of the specific detected object.

6. The method of claim 4 wherein the relevant audio or visual effect is an audio track that has a characteristic frequency that is substantially a multiple or fractional value of a frequency of motion of the specific detected object.

7. The method of claim 1 , wherein the suitability criterion is a threshold number of motion characteristics that meet the predetermined requirement.

8. The method of claim 7 wherein the predetermined requirement for the motion characteristics is a speed of motion of the specific detected object.

9. The method of claim 1 wherein the predetermined requirement for the specific object motion characteristic is a collision of the specific detected object with another object.

10. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to perform operations to generate a modified video file, the operations comprising:

initiating scanning of an original video file;

detecting objects represented in the original video file using computer vision object-detection techniques;

determining object motion characteristics for the detected objects;

based on the object motion characteristics not meeting a suitability criterion, terminating the method;

based on the object motion characteristics meeting a suitability criterion, performing the following steps;

based on a specific object motion characteristic for a specific detected object meeting a predetermined requirement, selecting a relevant audio or visual effect;

applying the relevant audio or visual effect to the original video file to create the modified video file; and

providing a notification of the existence of the modified video file on a display.

11. The computer-readable storage medium of claim 10 wherein the relevant audio or visual effect is a sound effect applied to correspond to a change in direction of the specific detected object.

12. The computer-readable storage medium of claim 10 wherein the predetermined requirement for the specific object motion characteristic is repeating motion of the specific detected object.

13. The computer-readable storage medium of claim 12 wherein the relevant audio or visual effect is a sound effect that is applied repetitively at a frequency of motion of the specific detected object.

14. The computer-readable storage medium of claim 12 wherein the suitability criterion is a threshold number of motion characteristics that meet the predetermined requirement.

15. A computing apparatus comprising:

a processor; and

a memory storing instructions that, when executed by the processor, configure the apparatus to generate a modified video file by:

initiating scanning of an original video file;

detecting objects represented in the original video file using computer vision object-detection techniques;

determining object motion characteristics for the detected objects;

based on the object motion characteristics not meeting a suitability criterion, terminating the method;

based on the object motion characteristics meeting a suitability criterion, performing the following steps;

based on a specific object motion characteristic for a specific detected object meeting a predetermined requirement, selecting a relevant audio or visual effect;

applying the relevant audio or visual effect to the original video file to create the modified video file; and

providing a notification of the existence of the modified video file on a display.

16. The computing apparatus of claim 15 wherein the predetermined requirement for the specific object motion characteristic is a change in direction of the specific detected object.

17. The computing apparatus of claim 15 wherein the predetermined requirement for the specific object motion characteristic is a collision of the specific detected object with another object.

18. The computing apparatus of claim 15 wherein the predetermined requirement for the specific object motion characteristic is a frequency of motion of the specific detected object.

19. The computing apparatus of claim 15 wherein the suitability criterion is a threshold number of motion characteristics that meet the predetermined requirement.

20. The computing apparatus of claim 19 wherein the predetermined requirement for the motion characteristics is a speed of motion of the specific detected object.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 13, 2022
From: SPREITZER, KENNETH JAY; WU, HAOYUN
To: SNAP INC.
Reel/Frame 062065/0741 →
Continuity (2)
Provisional Application 63200539 · Mar 12, 2021
Related Publication 20220293133A1 · Sep 15, 2022