IP Library Granted Patent US 10,923,154
Granted Patent B2
US 10,923,154 · App. 16/844,488 · Granted Feb 16, 2021

Systems and methods for determining highlight segment sets

Inventors: Desmond Chik (Mountain View, CA); Daniel Tse (San Mateo, CA); Jonathan Wills (San Mateo, CA)
Assignee: GoPro, Inc.
G11B27/034G06K9/0055G06K9/00718G06K9/00758G06K9/00765G06K9/4628G06K9/6271G11B27/11G06K9/6269
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,923,154
App. No.
16/844,488
Granted
Feb 16, 2021
Kind
B2
Abstract

A system and/or method configured to determine highlight segments. Content files that define content in a content segment set may be obtained. A highlight segment set may be determined from the content segment set. Determining the highlight segment set may include iterating (a)-(c) for multiple iterations. At (a), individual content segments included in the content segment set may be selected as a selected content segment for inclusion in the highlight segment set. At, (b) diversity scores for content segments that are (i) included in the content segment set and (ii) not yet selected for inclusion in the highlight segment set may be determined. At (c), one or more of the content segments may be disqualified for inclusion in the highlight segment set for future iterations based on the diversity scores.

Claims (40)

1. A system configured for determining highlight segments, the system comprising: one or more processors configured by machine-readable instructions to:

obtain content files that define a content segment set, the content segment set including a first content segment and a second content segment; and

iteratively select multiple content segments from the content segment set for inclusion in a highlight segment set by:

selecting an individual content segment included in the content segment set as a selected content segment for inclusion in the highlight segment set;

determining diversity scores for content segments that are (i) included in the content segment set and (ii) not selected for inclusion in the highlight segment set, the diversity scores indicating similarity between the individual content segments not selected for inclusion in the highlight segment set and the selected content segment; and

disqualifying one or more of the content segments from being selected for inclusion in the highlight segment set in future iterations based on the diversity scores.

2. The system of claim 1 , wherein the iterative selection of the multiple content segments for inclusion in the highlight segment set continues until a threshold number of content segments are selected for inclusion in the highlight segment set.

3. The system of claim 1 , wherein the iterative selection of the multiple content segments for inclusion in the highlight segment set continues until there are no content segments remaining in the content segment set that are not disqualified or included in the highlight segment set.

4. The system of claim 1 , wherein the diversity scores are determined based on one or more of visual similarity, temporal similarity, and/or semantic similarity between the individual content segments not selected for inclusion in the highlight segment set and the selected content segment, such that for a first iteration of the iterative selection of the multiple content segments for inclusion in the highlight segment set, a first diversity score is determined based on one or more of the visual similarity, the temporal similarity, and/or the semantic similarity between the second content segment and the first content segment.

5. The system of claim 4 , wherein the diversity scores are further determined based on emotive similarity between the individual content segments not selected for inclusion in the highlight segment set and the selected content segment, such that for the first iteration of the iterative selection of the multiple content segments for inclusion in the highlight segment set, the first diversity score is further determined based on the emotive similarity between the second content segment and the first content segment.

6. The system of claim 1 , wherein the second content segment is disqualified at the first iteration based on a first diversity score determined for the second content segment being below a threshold diversity score, the first diversity score indicating similarity between the second content segment and the first content segment.

7. The system of claim 1 , wherein the content segment set further includes a third content segment and a fourth content segment, and determining diversity scores includes determining a first diversity score for the second content segment, determining a second diversity score for the third content segment, and determining a third diversity score for the fourth content segment,

wherein the first diversity score for the second content segment is determined based on one or more of the visual similarity, the temporal similarity, and/or the semantic similarity between the second content segment and the first content segment, the second diversity score for the third content segment is determined based on one or more of the visual similarity, the temporal similarity, and/or the semantic similarity between the third content segment and the first content segment, and the third diversity score for the fourth content segment is determined based on one or more of the visual similarity, the temporal similarity, and/or the semantic similarity between the fourth content segment and the first content segment.

8. The system of claim 7 , wherein responsive to the first diversity score and the second diversity score being below a threshold diversity score, the one or more processors are further configured by machine-readable instructions to disqualify the second content segment and the third content segment for inclusion in the highlight segment set for future iterations.

9. The system of claim 1 , wherein the one or more processors are configured by machine-readable instructions to:

generate a graphical representation of attributes of one or more of the second content segment, a third content segment, and/or a fourth content segment based on one or more of a first diversity score determined for the second content segment, a second diversity score determined for the third content segment, and a third diversity score determined for the fourth content segment;

wherein one or more of the content segments are disqualified for inclusion in the highlight segment set for future iterations based on the graphical representation.

10. The system of claim 1 , wherein the second content segment is disqualified during a first iteration of the iterative selection of the multiple content segments for inclusion in the highlight segment set based on a first diversity score determined for the second content segment being less than other diversity scores for other content segments in the content segment set, wherein the first diversity score for the first iteration indicates similarity between the second content segment and the first content segment.

11. The system of claim 1 , wherein the individual content segment included in the content segment set is selected as the selected content segment for inclusion in the highlight segment set based on a selection criterion, the selection criterion including a quality score.

12. The system of claim 1 , wherein the content segment set includes one or more video segments, images, and/or audio segments.

13. A method for determining highlight segments, the method being implemented in a computer system that includes one or more processors configured by machine-readable instructions, the method comprising:

obtaining content files that define a content segment set, the content segment set including a first content segment and a second content segment; and

iteratively selecting multiple content segments from the content segment set for inclusion in a highlight segment set by:

selecting an individual content segment included in the content segment set as a selected content segment for inclusion in the highlight segment set;

determining diversity scores for content segments that are (i) included in the content segment set and (ii) not selected for inclusion in the highlight segment set, the diversity scores indicating similarity between the individual content segments not selected for inclusion in the highlight segment set and the selected content segment; and

disqualifying one or more of the content segments from being selected for inclusion in the highlight segment set in future iterations based on the diversity scores.

14. The method of claim 13 , wherein the iterative selection of the multiple content segments for inclusion in the highlight segment set continues until a threshold number of content segments are selected for inclusion in the highlight segment set.

15. The method of claim 13 , wherein the iterative selection of the multiple content segments for inclusion in the highlight segment set continues until there are no content segments remaining in the content segment set that are not disqualified or included in the highlight segment set.

16. The method of claim 13 , wherein the diversity scores are determined based on one or more of visual similarity, temporal similarity, and/or semantic similarity between the individual content segments yet selected for inclusion in the first highlight segment set and the selected content segment, such that for a first iteration of the iterative selection of the multiple content segments for inclusion in the highlight segment set, a first diversity score is determined based on one or more of the visual similarity, the temporal similarity, and/or the semantic similarity between the second content segment and the first content segment.

17. The method of claim 16 , wherein the diversity scores are further determined based on emotive similarity between the individual content segments not selected for inclusion in the highlight segment set and the selected content segment, such that for the first iteration of the iterative selection of the multiple content segments for inclusion in the highlight segment set, a first diversity score is further determined based on the emotive similarity between the second content segment and the first content segment.

18. The method of claim 13 , wherein the second content segment is disqualified at the first iteration based on a first diversity score determined for the second content segment being below a threshold diversity score, the first diversity score indicating similarity between the second content segment and the first content segment.

19. The method of claim 13 , wherein the content segment set further includes a third content segment and a fourth content segment, and determining diversity scores includes determining a first diversity score for the second content segment, determining a second diversity score for the third content segment, and determining a third diversity score for the fourth content segment,

wherein the first diversity score for the second content segment is determined based on one or more of the visual similarity, the temporal similarity, and/or the semantic similarity between the second content segment and the first content segment, the second diversity score for the third content segment is determined based on one or more of the visual similarity, the temporal similarity, and/or the semantic similarity between the third content segment and the first content segment, and wherein the third diversity score for the fourth content segment is determined based on one or more of the visual similarity, the temporal similarity, and/or the semantic similarity between the fourth content segment and the first content segment.

20. The method of claim 19 , wherein responsive to the first diversity score and the second diversity score being below a threshold diversity score, disqualifying one or more the content segments includes disqualifying the second content segment and the third content segment for inclusion in the highlight segment set for future iterations.

21. The method of claim 13 , further comprising:

generating a graphical representation of attributes of one or more of the second content segment, a third content segment, and/or a fourth content segment based on one or more of a first diversity score determined for the second content segment, a second diversity score determined for the third content segment, and a third diversity score determined for the fourth content segment;

wherein one or more of the content segments are disqualified for inclusion in the highlight segment set for future iterations based on the graphical representation.

22. The method of claim 13 , wherein the second content segment is disqualified during a first iteration of the iterative selection of the multiple content segments for inclusion in the highlight segment set based on a first diversity score determined for the second content segment being less than other diversity scores for other content segments in the content segment set, wherein the first diversity score for the first iteration indicates similarity between the second content segment and the first content segment.

23. The method of claim 13 , wherein the individual content segment included in the content segment set is selected as the selected content segment for inclusion in the highlight segment set based on a selection criterion, the selection criterion including a quality score.

24. The method of claim 13 , wherein the content segment set includes one or more video segments, images, and/or audio segments.

Assignments (5)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
SECURITY INTEREST Recorded Oct 19, 2020
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 054113/0594 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 9, 2020
From: CHIK, DESMOND; TSE, DANIEL; WILLS, JONATHAN
To: GOPRO, INC.
Reel/Frame 052357/0637 →
Continuity (3)
Continuation 16011014 · Jun 18, 2018
Continuation 15295862 · Oct 17, 2016
Related Publication 20200234734A1 · Jul 23, 2020