IP Library Granted Patent US 10,726,272
Granted Patent B2
US 10,726,272 · App. 15/959,954 · Granted Jul 28, 2020

Systems and methods for generating a video summary

Inventors: Jonathan Wills (San Mateo, CA); Daniel Tse (San Mateo, CA); Desmond Chik (Mountain View, CA); Brian Schunck (San Diego, CA)
Assignee: Go Pro, Inc.
G06K9/00751G06K9/66G11B27/031G11B27/3081H04N5/225
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,726,272
App. No.
15/959,954
Granted
Jul 28, 2020
Kind
B2
Abstract

Systems and method of generating video summaries are presented herein. Information defining a video may be obtained. The video may include a set of frame images. Parameter values for parameters of individual frame images of the video may be determined. Interest weights for the frame images may be determined. An interest curve for the video that characterizes the video by interest weights as a function of progress through the set of frame images may be generated. One or more curve attributes of the interest curve may be identified and one or more interest curve values of the interest curve that correspond to individual curve attributes may be determined. Interest curve values of the interest curve may be compared to threshold curve values. A subset of frame images of the video to include within a video summary of the video may be identified based on the comparison.

Claims (35)

1. A system configured to generate a video summary, the system comprising:

one or more physical computer processors configured by computer readable instructions to:

obtain information defining a video, the video including a set of images;

determine individual parameter values for multiple parameters of individual images of the video, the parameter values including a set of parameter values for a parameter of the individual images in the set of images;

determine individual interest weights of the individual images based on the individual parameter values of the multiple parameters of the individual images, the interest weights including a set of interest weights for the set of images determined from the set of parameter values, wherein determining a given interest weight of a given image comprises determining impact of individual ones of multiple parameter values on the given interest weight;

generate an interest curve for the video based on the individual interest weights of the individual images, the interest curve characterizing the video based on the interest weights as a function of progress through the set of images, the interest curve comprising a set of interest curve values that correspond to the interest weights in the set of interest weights and the images in the set of images;

identify one or more curve attributes of the interest curve, the one or more curve attributes including a first curve attribute, the first curve attribute corresponding to a first set of interest curve values and a subset of the images;

identify the subset of images of the video to include within the video summary of the video based on the first curve attribute and the first set of interest curve values; and

generate the video summary using the subset of images.

2. The system of claim 1 , wherein the subset of images of the video are identified further based on the first set of interest curve values being greater than one or more threshold curve values corresponding to the first curve attribute.

3. The system of claim 2 , wherein the one or more threshold curve values are specific to content of the video.

4. The system of claim 1 , wherein the one or more curve attributes include one or more of an overall shape, an infection point, a maximum, a minimum, a rate of change, or a rate of the rate of change of the interest curve.

5. The system of claim 1 , wherein the multiple parameters include one or more of a scene parameter, a quality parameter, a capture parameter, or an audio parameter.

6. The system of claim 5 , wherein the scene parameter specifies one or more scene features of the individual images, the one or more scene features including one or more of feature points, objects, faces, colors, scene compilation, or text of the individual images.

7. The system of claim 6 , wherein the quality parameter specifics one or more quality attributes of the individual images, the one or more quality attributes including one or more of blurriness, glare, saturation, brightness, contrast, sharpness, framing of an object of interest, of a representativeness of the individual images with respect to an overall context of the video.

8. The system of claim 1 , wherein determining the impact of individual ones of the multiple parameter values on the given interest weight is based on a convolutional neural network.

9. The system of claim 1 , wherein the information defining the video includes metadata associated with the video, and the individual parameter values for multiple parameters of the individual images of the video are determined further based on the metadata.

10. The system of claim 9 , wherein metadata associated with the video includes one or more of capture settings of a capture device, sensor output of one or more sensors coupled to the capture device, or user-provided information.

11. A method of generating a video summary, the method being implemented in a computer system comprising one or more physical processors and storage media storing machine-readable instructions, the method comprising:

obtaining, by the computing system, information defining a video, the video including a set of images;

determining, by the computing system, individual parameter values for multiple parameters of individual images of the video, the parameter values including a set of parameter values for a parameter of the individual images in the set of images;

determining, by the computing system, interest weights of the individual images based on the individual parameter values of the multiple parameters of the individual images, the interest weights including a set of interest weights for the set of images determined from the set of parameter values, wherein determining a given interest weight of a given image comprises determining impact of individual ones of multiple parameter values on the given interest weight;

generating, by the computing system, an interest curve for the video based on the individual interest weights of the individual images, the interest curve characterizing the video based on the interest weights as a function of progress through the set of images, the interest curve comprising a set of interest curve values that correspond to the interest weights in the set of interest weights and the images in the set of images;

identifying, by the computing system, one or more curve attributes of the interest curve, the one or more curve attributes including a first curve attribute, the first curve attribute corresponding to a first set of interest curve values and a subset of the images;

identifying, by the computing system, the subset of images of the video to include within the video summary of the video based on the first curve attribute and the first set of interest curve values; and

generating, by the computing system, the video summary using the subset of images.

12. The method of claim 11 , wherein the subset of images of the video are identified further based on the first set of interest curve values being greater than one or more threshold curve values corresponding to the first curve attribute.

13. The method of claim 12 , wherein the one or more threshold curve values are specific to content of the video.

14. The method of claim 11 , wherein the one or more curve attributes include one or more of an overall shape, an infection point, a maximum, a minimum, a rate of change, or a rate of the rate of change of the interest curve.

15. The method of claim 11 , wherein the multiple parameters include one or more of a scene parameter, a quality parameter, a capture parameter, or an audio parameter.

16. The method of claim 15 , wherein the scene parameter specifies one or more scene features of the individual images, the one or more scene features including one or more of feature points, objects, faces, colors, scene compilation, or text of the individual images.

17. The method of claim 16 , wherein the quality parameter specifics one or more quality attributes of the individual images, the one or more quality attributes including one or more of blurriness, glare, saturation, brightness, contrast, sharpness, framing of an object of interest, of a representativeness of the individual images with respect to an overall context of the video.

18. The method of claim 11 , wherein determining the impact of individual ones of the multiple parameter values on the given interest weight is based on a convolutional neural network.

19. The method of claim 11 , wherein the information defining the video includes metadata associated with the video, and the individual parameter values for multiple parameters of the individual images of the video are determined further based on the metadata.

20. The method of claim 19 , wherein metadata associated with the video includes one or more of capture settings of a capture device, sensor output of one or more sensors coupled to the capture device, or user-provided information.

Assignments (6)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
SECURITY INTEREST Recorded Oct 19, 2020
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 054113/0594 →
SECURITY INTEREST Recorded Sep 5, 2018
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 047016/0417 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 23, 2018
From: WILLS, JONATHAN; TSE, DANIEL; CHIK, DESMOND; SCHUNCK, BRIAN
To: GOPRO, INC.
Reel/Frame 045612/0738 →
Continuity (2)
Continuation 15244690 · Aug 23, 2016
Related Publication 20180247130A1 · Aug 30, 2018