IP Library Granted Patent US 9,916,863
Granted Patent B1
US 9,916,863 · App. 15/442,050 · Granted Mar 13, 2018

Systems and methods for editing videos based on shakiness measures

Inventors: Tom Medioni (Paris, FR); Vincent Garcia (Issy-les-Moulineaux, FR)
Assignee: GoPro, Inc.
G11B27/031G06K9/00751G06K9/00758H04N5/144H04N9/646
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,916,863
App. No.
15/442,050
Granted
Mar 13, 2018
Kind
B1
Abstract

Video information defining video content may be accessed. The video content may include video frames. Motion vectors for the video frames may be determined. The motion vectors may represent motion of one or more visuals captured within individual video frames. A transformation matrix for the video frames may be determined based on the motion vectors. The transformation matrix may characterize rigid transformations between pairs of the video frames. Shakiness metrics for the video frames may be determined based on the transformation matrix. A shakiness threshold may be obtained. One or more of the video frames may be identified based on the shakiness metrics, the shakiness threshold, and algorithms with hysteresis or finite-state machines. A video summary of the video content may be generated. The video summary may include the one or more identified video frames.

Claims (45)

1. A system for editing videos based on movement captured within the videos, the system comprising:

physical storage media storing video information defining video content, the video content including video frames;

one or more physical processors configured by machine readable instructions to:

access the video information defining the video content;

determine motion vectors for the video frames, the motion vectors representing motion of one or more visuals captured within individual video frames;

determine a transformation matrix for the video frames based on the motion vectors, the transformation matrix characterizing rigid transformations between pairs of the video frames;

determine shakiness metrics for the video frames based on the transformation matrix;

obtain a shakiness threshold; and

identify one or more of the video frames based on the shakiness metrics and the shakiness threshold;

generate a video summary of the video content, the video summary including the one or more identified video frames.

2. The system of claim 1 , wherein the shakiness metrics for the video frames are determined based on a rate of change in a lateral translation, a rate of change in a vertical translation, a rate of change in an angular translation, and a rate of change in a linear first-order approximation of a scaling factor of the transformation matrix.

3. The system of claim 2 , wherein the rate of change in the lateral translation is normalized based on a width of the video content, the rate of change in the vertical translation is normalized based on a height of the video content, the rate of change in the angular translation is matched to the lateral translation and the vertical translation via a first matching factor, and the rate of change in the linear first-order approximation of the scaling factor is matched to the lateral translation and the vertical translation via a second matching factor.

4. The system of claim 1 , wherein the rigid transformations are estimated using a RANSAC algorithm.

5. The system of claim 1 , wherein the motion vectors for the video frames are determined based on movement of a pixel group.

6. The system of claim 5 , wherein the pixel group includes an eight-by-eight pixel block.

7. The system of claim 1 , wherein the one or more physical processors are further configured by machine readable instruction to apply one or more filters to remove noise from the video frames.

8. The system of claim 7 , wherein the video content is captured at thirty frames per second, and the one or more filters are applied using a time scale of fives frames.

9. The system of claim 7 , wherein the video content is captured at sixty frames per second, and the one or more filters are applied using a time scale of ten frames.

10. A method for editing videos based on movement captured within the videos, the method implemented in a system including one or more physical processors, the method comprising:

accessing, by the one or more physical processors, video information defining video content, the video content including video frames;

determining, by the one or more physical processors, motion vectors for the video frames, the motion vectors representing motion of one or more visuals captured within individual video frames;

determining, by the one or more physical processors, a transformation matrix for the video frames based on the motion vectors, the transformation matrix characterizing rigid transformations between pairs of the video frames;

determining, by the one or more physical processors, shakiness metrics for the video frames based on the transformation matrix;

obtaining, by the one or more physical processors, a shakiness threshold; and

identifying, by the one or more physical processors, one or more of the video frames based on the shakiness metrics and the shakiness threshold;

generating, by the one or more physical processors, a video summary of the video content, the video summary including the one or more identified video frames.

11. The method of claim 10 , wherein the shakiness metrics for the video frames are determined based on a rate of change in a lateral translation, a rate of change in a vertical translation, a rate of change in an angular translation, and a rate of change in a linear first-order approximation of a scaling factor of the transformation matrix.

12. The method of claim 11 , wherein the rate of change in the lateral translation is normalized based on a width of the video content, the rate of change in the vertical translation is normalized based on a height of the video content, the rate of change in the angular translation is matched to the lateral translation and the vertical translation via a first matching factor, and the rate of change in the linear first-order approximation of the scaling factor is matched to the lateral translation and the vertical translation via a second matching factor.

13. The method of claim 10 , wherein the rigid transformations are estimated using a RANSAC algorithm.

14. The method of claim 10 , wherein the motion vectors for the video frames are determined based on movement of a pixel group.

15. The method of claim 14 , wherein the pixel group includes an eight-by-eight pixel block.

16. The method of claim 10 , further comprising applying, by the one or more physical processors, one or more filters to remove noise from the video frames.

17. The method of claim 16 , wherein the video content is captured at thirty frames per second, and the one or more filters are applied using a time scale of fives frames.

18. The method of claim 16 , wherein the video content is captured at sixty frames per second, and the one or more filters are applied using a time scale of ten frames.

19. A system for editing videos based on movement captured within the videos, the system comprising:

physical storage media storing video information defining video content, the video content including video frames;

one or more physical processors configured by machine readable instructions to:

access the video information defining the video content;

determine motion vectors for the video frames, the motion vectors representing motion of one or more visuals captured within individual video frames, wherein the motion vectors for the video frames are determined based on movement of an eight-by-eight pixel block;

determine a transformation matrix for the video frames based on the motion vectors, the transformation matrix characterizing rigid transformations between pairs of the video frames;

determine shakiness metrics for the video frames based on the transformation matrix, wherein the shakiness metrics for the video frames are determined based on a rate of change in a lateral translation, a rate of change in a vertical translation, a rate of change in an angular translation, and a rate of change in a linear first-order approximation of a scaling factor of the transformation matrix;

obtain a shakiness threshold; and

identify one or more of the video frames based on the shakiness metrics and the shakiness threshold;

generate a video summary of the video content, the video summary including the one or more identified video frames.

20. The system of claim 19 , wherein the rate of change in the lateral translation is normalized based on a width of the video content, the rate of change in the vertical translation is normalized based on a height of the video content, the rate of change in the angular translation is matched to the lateral translation and the vertical translation via a first matching factor, and the rate of change in the linear first-order approximation of the scaling factor is matched to the lateral translation and the vertical translation via a second matching factor.

Assignments (7)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
CORRECTIVE ASSIGNMENT TO CORRECT THE SCHEDULE TO REMOVE APPLICATION 15387383 AND REPLACE WITH 15385383 PREVIOUSLY RECORDED ON REEL 042665 FRAME 0065. ASSIGNOR(S) HEREBY CONFIRMS THE SECURITY INTEREST. Recorded Oct 23, 2019
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 050808/0824 →
SECURITY INTEREST Recorded Sep 5, 2018
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 047016/0417 →
SECURITY INTEREST Recorded Jun 1, 2017
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 042665/0065 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 24, 2017
From: MEDIONI, TOM; GARCIA, VINCENT
To: GOPRO, INC.
Reel/Frame 041373/0025 →