IP Library › Granted Patent US 12,457,382
Granted Patent B2
US 12,457,382 · App. 18/546,386 · Granted Oct 28, 2025

Identifying and labeling segments within video content

Inventors: Amanmeet Garg (Santa Clara, CA); Sharmishtha Gupta (Fremont, CA); Andreas Schmidt (San Pablo, CA); Lakshika Balasuriya (Walnut Creek, CA); Aneesh Vartakavi (Emeryville, CA)
Assignee: Gracenote, Inc.
H04N21/44008G06F18/253G06V20/41H04N21/8352
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,457,382
App. No.
18/546,386
Granted
Oct 28, 2025
Kind
B2
Abstract

In one aspect, an example method includes (i) obtaining fingerprint repetition data for a portion of video content, with the fingerprint repetition data including a list of other portions of video content matching the portion of video content and respective reference identifiers for the other portions of video content; (ii) identifying the portion of video content as a program segment rather than an advertisement segment based at least on a number of unique reference identifiers within the list of other portions of video content relative to a total number of reference identifiers within the list of other portions of video content; (iii) determining that the portion of video content corresponds to a program specified in an electronic program guide using a time stamp of the portion of video content; and (iv) storing an indication of the portion of video content in a data file for the program.

Claims (50)

1. A method comprising:

obtaining, by a computing system, fingerprint repetition data for a portion of video content, wherein the fingerprint repetition data comprises a list of other portions of video content matching the portion of video content and respective reference identifiers for the other portions of video content, each reference identifier corresponding to a respective different time the other portion of video content was presented;

identifying, by the computing system, the portion of video content as a program segment rather than an advertisement segment based at least on a number of unique reference identifiers within the list of other portions of video content relative to a total number of reference identifiers within the list of other portions of video content, wherein the total number of reference identifiers within the list of other portions of video content further includes a plurality of reference identifiers that are the same; and

based at least on the identifying of the portion of video content as a program segment, storing, by the computing system, an indication of the portion of video content in a data file for a corresponding program.

2. The method of claim 1 , wherein obtaining the fingerprint repetition data comprises searching for matches to fingerprints of the portion of video content within a video database so as to obtain the list of other portions of video content.

3. The method of claim 1 , wherein identifying the portion of video content as a program segment rather than an advertisement segment based at least on the number of unique reference identifiers relative to the total number of reference identifiers comprises determining that a ratio of the number of unique reference identifiers to the total number of reference identifiers satisfies a threshold.

4. The method of claim 1 , further comprising obtaining logo coverage data for the portion of video content,

wherein the logo coverage data is indicative of a percent of time that a logo overlays the portion of video content, and

wherein the identifying the portion of video content as a program segment rather than an advertisement segment is further based on the logo coverage data.

5. The method of claim 4 , wherein the identifying the portion of video content as a program segment rather than an advertisement segment is further based on a number of portions of video content in the list of other portions of video content and a length of the portion of video content.

6. The method of claim 1 , further comprising:

obtaining transition data for a section of video content that includes the portion of video content; and

identifying boundaries of the portion of video content using the transition data.

7. The method of claim 1 , further comprising after identifying the portion of video content as a program segment, merging the portion of video content with an adjacent portion of video content that is identified as a program segment.

8. The method of claim 7 , wherein merging the portion of video content with the adjacent portion of video content comprises:

obtaining a first list of matching portions for the portion of video content;

obtaining a second list of matching portions for the adjacent portion of video content;

identifying correspondences between the first list and the second list using timestamps for matching portions of the first list of matching portions and timestamps for matching portions of the second list of matching portions; and

based on the correspondences, merging the portion of video content with the adjacent portion of video content.

9. The method of claim 1 , further comprising generating a copy of the program using the data file for the program.

10. The method of claim 1 , further comprising obtaining closed captioning repetition data for the portion of video content,

wherein the identifying the portion of video content as a program segment rather than an advertisement segment is further based on the closed captioning repetition data.

11. The method of claim 10 , further comprising:

generating features using the closed captioning repetition data; and

providing the features as input to a classification model, wherein the classification model is configured to output classification data indicative of a likelihood of the features being characteristic of a program segment,

wherein the identifying the portion of video content as a program segment rather than an advertisement segment is further based on the classification data.

12. The method of claim 1 , further comprising:

obtaining fingerprint repetition data for another portion of video content;

identifying the other portion of video content as an advertisement segment rather than a program segment using the fingerprint repetition data;

identifying metadata for the other portion of video content; and

storing an indication of the other portion of video content and the metadata in another data file.

13. A non-transitory computer-readable medium having stored thereon program instructions that upon execution by a processor, cause performance of a set of acts comprising:

obtaining fingerprint repetition data for a portion of video content, wherein the fingerprint repetition data comprises a list of other portions of video content matching the portion of video content and respective reference identifiers for the other portions of video content, each reference identifier corresponding to a respective different time the other portion of video content was presented;

identifying the portion of video content as a program segment rather than an advertisement segment based at least on a number of unique reference identifiers within the list of other portions of video content relative to a total number of reference identifiers within the list of other portions of video content, wherein the total number of reference identifiers within the list of other portions of video content further includes a plurality of reference identifiers that are the same; and

based at least on the identifying of the portion of video content as a program segment, storing an indication of the portion of video content in a data file for a corresponding program.

14. The non-transitory computer-readable medium of claim 13 , wherein obtaining the fingerprint repetition data comprises searching for matches to fingerprints of the portion of video content within a video database so as to obtain the list of other portions of video content.

15. The non-transitory computer-readable medium of claim 13 , wherein identifying the portion of video content as a program segment rather than an advertisement segment based at least on the number of unique reference identifiers relative to the total number of reference identifiers comprises determining that a ratio of the number of unique reference identifiers to the total number of reference identifiers satisfies a threshold.

16. A computing system configured for performing a set of acts comprising:

obtaining fingerprint repetition data for a portion of video content, wherein the fingerprint repetition data comprises a list of other portions of video content matching the portion of video content and respective reference identifiers for the other portions of video content, each reference identifier corresponding to a respective different time the other portion of video content was presented;

identifying the portion of video content as a program segment rather than an advertisement segment based at least on a number of unique reference identifiers within the list of other portions of video content relative to a total number of reference identifiers within the list of other portions of video content, wherein the total number of reference identifiers within the list of other portions of video content further includes a plurality of reference identifiers that are the same; and

based at least on the identifying of the portion of video content as a program segment, storing an indication of the portion of video content in a data file for a corresponding program.

17. The computing system of claim 16 , wherein obtaining the fingerprint repetition data comprises searching for matches to fingerprints of the portion of video content within a video database so as to obtain the list of other portions of video content.

18. The computing system of claim 16 , wherein identifying the portion of video content as a program segment rather than an advertisement segment based at least on the number of unique reference identifiers relative to the total number of reference identifiers comprises determining that a ratio of the number of unique reference identifiers to the total number of reference identifiers satisfies a threshold.

19. The computing system of claim 16 , wherein:

the set of acts further comprises obtaining logo coverage data for the portion of video content,

the logo coverage data is indicative of a percent of time that a logo overlays the portion of video content, and

the identifying the portion of video content as a program segment rather than an advertisement segment is further based on the logo coverage data.

20. The computing system of claim 16 , wherein the set of acts further comprises:

obtaining transition data for a section of video content that includes the portion of video content; and

identifying boundaries of the portion of video content using the transition data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 14, 2023
From: GARG, AMANMEET; GUPTA, SHARMISHTHA; SCHMIDT, ANDREAS; BALASURIYA, LAKSHIKA; VARTAKAVI, ANEESH
To: GRACENOTE, INC.
Reel/Frame 064582/0235 →
Continuity (3)
Continuation 17401656 · Aug 13, 2021
Provisional Application 63150023 · Feb 16, 2021
Related Publication 20240163508A1 · May 16, 2024
References Cited (27)
US 9665789B2 · Lee · 2017 [cited by applicant]
US 10581541B1 · Nelson et al. · 2020 [cited by applicant]
US 12132953B2 · Garg · 2024 [cited by examiner]
US 20030091237A1 · Cohen-Solal et al. · 2003 [cited by applicant]
US 20050025358A1 · Qi et al. · 2005 [cited by applicant]
US 20090256972A1 · Ramaswamy et al. · 2009 [cited by applicant]
US 20120150928A1 · Ward et al. · 2012 [cited by applicant]
US 20120167133A1 · Carroll et al. · 2012 [cited by applicant]
US 20130247083A1 · Impollonia et al. · 2013 [cited by applicant]
US 20140089424A1 · Oztaskent et al. · 2014 [cited by applicant]
US 20140282671A1 · McMillan · 2014 [cited by applicant]
US 20160110877A1 · Schwartz et al. · 2016 [cited by applicant]
US 20160226942A1 · Bhardwaj et al. · 2016 [cited by applicant]
US 20170070774A1 · Bilobrov et al. · 2017 [cited by applicant]
US 20180007405A1 · McLean et al. · 2018 [cited by applicant]
US 20180121541A1 · Kalampoukas et al. · 2018 [cited by applicant]
US 20180199094A1 · Jeong et al. · 2018 [cited by applicant]
US 20210117685A1 · Sureshkumar et al. · 2021 [cited by applicant]
US 20210151034A1 · Hasan et al. · 2021 [cited by applicant]
US 20210377603A1 · Wright et al. · 2021 [cited by applicant]
US 20220019809A1 · Younessian et al. · 2022 [cited by applicant]
KR 20160053549 · 2016 [cited by applicant]
KR 20180082427 · 2018 [cited by applicant]
Office Action from U.S. Appl. No. 17/496,297, dated Oct. 14, 2022. [cited by applicant]
International Preliminary Report on Patentability from International Application No. PCT/US2022/013240, dated Aug. 29, 2023. [cited by applicant]
International Search Report issued in International Application No. PCT/US2022/013239, mailed May 3, 2022, 3 pages. [cited by applicant]
Written Opinion issued in International Application No. PCT/US2022/013239, mailed May 3, 2022, 5 pages. [cited by applicant]