IP Library Granted Patent US 10,643,661
Granted Patent B2
US 10,643,661 · App. 16/011,014 · Granted May 5, 2020

Systems and methods for determining highlight segment sets

Inventors: Desmond Chik (Mountain View, CA); Daniel Tse (San Mateo, CA); Jonathan Wills (San Mateo, CA)
Assignee: GoPro, Inc.
G11B27/034G06K9/0055G06K9/00718G06K9/00758G06K9/00765G06K9/4628G06K9/6271G11B27/11G06K9/6269
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,643,661
App. No.
16/011,014
Granted
May 5, 2020
Kind
B2
Abstract

A system and/or method configured to determine highlight segments. Content files that define content in a content segment set may be obtained. A highlight segment set may be determined from the content segment set. Determining the highlight segment set may include iterating (a)-(c) for multiple iterations. At (a), individual content segments included in the content segment set may be selected as a selected content segment for inclusion in the highlight segment set. At, (b) diversity scores for content segments that are (i) included in the content segment set and (ii) not yet selected for inclusion in the highlight segment set may be determined. At (c), one or more of the content segments may be disqualified for inclusion in the highlight segment set for future iterations based on the diversity scores.

Claims (43)

1. A system configured for determining highlight segments, the system comprising:

one or more processors configured by machine-readable instructions to:

obtain content files that define content in a content segment set, the content segment set including a first content segment and a second content segment;

determine a highlight segment set from the content segment set by:

(a) selecting an individual content segment included in the content segment set as a selected content segment for inclusion in the highlight segment set;

(b) determining diversity scores for content segments that are (i) included in the content segment set and (ii) not yet selected for inclusion in the highlight segment set, the diversity scores indicating a level of similarity between the individual content segments not yet selected for inclusion in the highlight segment set and the selected content segment selected at operation (a); and

(c) disqualifying one or more of the content segments for inclusion in the highlight segment set for future iterations based on the diversity scores,

iterate (a)-(c) to determine the highlight segment set.

2. The system of claim 1 , wherein the one or more processors are further configured by machine-readable instructions to iterate (a)-(c) until a threshold number of content segments are selected for inclusion in the highlight segment set.

3. The system of claim 1 , wherein the one or more processors are further configured by machine-readable instructions to iterate (a)-(c) until there are no content segments remaining in the content segment set that are not disqualified or included in the highlight segment set.

4. The system of claim 1 , wherein the diversity scores are determined based on one or more of visual similarity, temporal similarity, and/or semantic similarity between the individual content segments not yet selected for inclusion in the highlight segment set and the selected content segment selected at operation (a), such that for a first iteration of (a)-(c), a first diversity score is determined based on one or more of the visual similarity, the temporal similarity, and/or the semantic similarity between the second content segment and the first content segment.

5. The system of claim 4 , wherein the diversity scores are further determined based on emotive similarity between the individual content segments not yet selected for inclusion in the highlight segment set and the selected content segment selected at operation (a), such that for the first iteration of (a)-(c), the first diversity score is further determined based on the emotive similarity between the second content segment and the first content segment.

6. The system of claim 1 , wherein the second content segment is disqualified at the first iteration based on a first diversity score determined for the second content segment being below a threshold diversity score, the first diversity score indicating the level of similarity between the second content segment and the first content segment.

7. The system of claim 1 , wherein the content segment set further includes a third content segment and a fourth content segment, and determining diversity scores for a first iteration of (b) includes determining a first diversity score for the second content segment, determining a second diversity score for the third content segment, and determining a third diversity score for the fourth content segment,

wherein the first diversity score for the second content segment is determined based on one or more of the visual similarity, the temporal similarity, and/or the semantic similarity between the second content segment and the first content segment, the second diversity score for the third content segment is determined based on one or more of the visual similarity, the temporal similarity, and/or the semantic similarity between the third content segment and the first content segment, and the third diversity score for the fourth content segment is determined based on one or more of the visual similarity, the temporal similarity, and/or the semantic similarity between the fourth content segment and the first content segment.

8. The system of claim 7 , wherein responsive to the first diversity score and the second diversity score being below a threshold diversity score for the first iteration, the one or more processors are further configured by machine-readable instructions to disqualify the second content segment and the third content segment for inclusion in the highlight segment set for future iterations.

9. The system of claim 1 , wherein the one or more processors are configured by machine-readable instructions to:

generate a graphical representation of attributes of one or more of the second content segment, a third content segment, and/or a fourth content segment based on one or more of a first diversity score determined for the second content segment, a second diversity score determined for the third content segment, and a third diversity score determined for the fourth content segment;

wherein one or more of the content segments are disqualified for inclusion in the highlight segment set for future iterations based on the graphical representation.

10. The system of claim 1 , wherein the second content segment is disqualified during a first iteration based on a first diversity score determined for the second content segment being less than other diversity scores for other content segments in the content segment set, wherein the first diversity score for the first iteration indicates the level of similarity between the second content segment and the first content segment.

11. The system of claim 1 , wherein the individual content segment included in the content segment set is selected as the selected content segment for inclusion in the highlight segment set based on a selection criterion, the selection criterion including a quality score.

12. The system of claim 1 , wherein the content segment set includes one or more video segments, images, and/or audio segments.

13. A method for determining highlight segments, the method being implemented in a computer system that includes one or more processors configured by machine-readable instructions, the method comprising:

obtaining content files that define content in a content segment set, the content segment set including a first content segment and a second content segment;

determining a highlight segment set from the content segment set by:

(a) selecting an individual content segment included in the content segment set as a selected content segment for inclusion in the highlight segment set;

(b) determining diversity scores for content segments that are (i) included in the content segment set and (ii) not yet selected for inclusion in the highlight segment set, the diversity scores indicating a level of similarity between the individual content segments not yet selected for inclusion in the highlight segment set and the selected content segment selected at operation (a); and

(c) disqualifying one or more of the content segments for inclusion in the highlight segment set for future iterations based on the diversity scores,

iterating (a)-(c) to determine the highlight segment set.

14. The method of claim 13 , further comprising iterating (a)-(c) until a threshold number of content segments are selected for inclusion in the highlight segment set.

15. The method of claim 13 , further comprising iterating (a)-(c) until there are no content segments remaining in the content segment set that are not disqualified or included in the highlight segment set.

16. The method of claim 13 , wherein the diversity scores are determined based on one or more of visual similarity, temporal similarity, and/or semantic similarity between the individual content segments not yet selected for inclusion in the first highlight segment set and the selected content segment selected at operation (a), such that for a first iteration of (a)-(c), a first diversity score is determined based on one or more of the visual similarity, the temporal similarity, and/or the semantic similarity between the second content segment and the first content segment.

17. The method of claim 16 , wherein the diversity scores are further determined based on emotive similarity between the individual content segments not yet selected for inclusion in the highlight segment set and the selected content segment selected at operation (a), such that for the first iteration of (a)-(c), a first diversity score is further determined based on the emotive similarity between the second content segment and the first content segment.

18. The method of claim 13 , wherein the second content segment is disqualified at the first iteration based on a first diversity score determined for the second content segment being below a threshold diversity score, the first diversity score indicating the level of similarity between the second content segment and the first content segment.

19. The method of claim 13 , wherein the content segment set further includes a third content segment and a fourth content segment, and determining diversity scores for a first iteration of (b) includes determining a first diversity score for the second content segment, determining a second diversity score for the third content segment, and determining a third diversity score for the fourth content segment,

wherein the first diversity score for the second content segment is determined based on one or more of the visual similarity, the temporal similarity, and/or the semantic similarity between the second content segment and the first content segment, the second diversity score for the third content segment is determined based on one or more of the visual similarity, the temporal similarity, and/or the semantic similarity between the third content segment and the first content segment, and wherein the third diversity score for the fourth content segment is determined based on one or more of the visual similarity, the temporal similarity, and/or the semantic similarity between the fourth content segment and the first content segment.

20. The method of claim 19 , wherein responsive to the first diversity score and the second diversity score being below a threshold diversity score, disqualifying one or more the content segments includes disqualifying the second content segment and the third content segment for inclusion in the highlight segment set for future iterations.

21. The method of claim 13 , further comprising:

generating a graphical representation of attributes of one or more of the second content segment, a third content segment, and/or a fourth content segment based on one or more of a first diversity score determined for the second content segment, a second diversity score determined for the third content segment, and a third diversity score determined for the fourth content segment;

wherein one or more of the content segments are disqualified for inclusion in the highlight segment set for future iterations based on the graphical representation.

22. The method of claim 13 , wherein the second content segment is disqualified during a first iteration based on a first diversity score determined for the second content segment being less than other diversity scores for other content segments in the content segment set, wherein the first diversity score for the first iteration indicates the level of similarity between the second content segment and the first content segment.

23. The method of claim 13 , wherein the individual content segment included in the content segment set is selected as the selected content segment for inclusion in the highlight segment set based on a selection criterion, the selection criterion including a quality score.

24. The method of claim 13 , wherein the content segment set includes one or more video segments, images, and/or audio segments.

Assignments (6)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
SECURITY INTEREST Recorded Oct 19, 2020
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 054113/0594 →
SECURITY INTEREST Recorded Sep 5, 2018
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 047016/0417 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 18, 2018
From: CHIK, DESMOND; TSE, DANIEL; WILLS, JONATHAN
To: GOPRO, INC.
Reel/Frame 046120/0381 →