IP Library › Granted Patent US 12,309,440
Granted Patent B2
US 12,309,440 · App. 16/230,830 · Granted May 20, 2025

Optimized ad placement based on automated video analysis and deep metadata extraction

Inventors: Basil Badawiyeh (Parker, CO); Vipul Patel (Great Falls, VA); Gary Schanman (Cherry Hills Village, CO)
Assignee: CHARTER COMMUNICATIONS OPERATING, LLC
H04N21/23424G06N20/00G06Q30/0269H04N21/2353H04N21/45457
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,309,440
App. No.
16/230,830
Filed
Dec 21, 2018
Granted
May 20, 2025
Kind
B2
Art Unit
2426
USPC
725/34
Abstract

In one aspect, an exemplary method for dynamic advertising insertion includes obtaining a video program comprising a sequence of segments; extracting metadata describing at least a specific one of the segments of the video program; based at least in part on the extracted metadata describing the specific segment, selecting advertising content corresponding to the specific segment; and generating an output video stream comprising the sequence of segments, wherein the selected advertising content is displayed in connection with the specific segment.

Claims (47)

1. A method for dynamic advertising insertion for a user, comprising:

obtaining a video program comprising a sequence of segments;

generating metadata describing at least a specific one of the segments of the video program;

building an enhanced user profile of the user based on the generated metadata;

based at least in part on the generated metadata describing the specific segment and the enhanced user profile built based on the generated metadata, selecting advertising content corresponding to the specific segment; and

generating an output video stream comprising the sequence of segments, wherein the selected advertising content is displayed to the user in connection with the specific segment;

wherein generating the metadata comprises at least scene detection using at least one of artificial intelligence and machine learning; and

wherein the selected advertising content is inserted into the output video stream between the specific segment and another segment;

further comprising determining, using the generated metadata, a preference of said user to watch said specific segment of the video program as opposed to the entire video program, wherein the profile is updated based at least in part on the preference and the specific segment is selected based on the preference of said user to watch said specific segment as opposed to the entire video program.

2. The method of claim 1 , wherein at least the specific segment comprises an image corresponding to a point of time within the video program.

3. The method of claim 1 , wherein at least the specific segment comprises a scene corresponding to a period of time within the video program.

4. The method of claim 1 , wherein the metadata indicates at least one object, person, location, event, or action shown within at least a portion of the specific segment.

5. The method of claim 1 , wherein the metadata indicates at least one sentiment or emotion corresponding to at least a portion of the specific segment.

6. The method of claim 1 , wherein selecting the advertising content comprises using the profile of the given user to anticipate an emotional response of the given user to the specific segment.

7. The method of claim 1 , wherein the profile of the given user comprises a probability that the given user supports a first team, further comprising selecting the advertising content to be favorable to the first team, despite the given user being geographically located in a region associated with a second team different than the first team.

8. The method of claim 1 , wherein the video program further comprises respective cues between the segments, and wherein a cue between the specific segment and another segment is replaced in the output video stream with the selected advertising content corresponding to the specific segment.

9. The method of claim 1 , wherein extracting the metadata describing at least the specific segment comprises determining a time within the video program at which the specific segment ends.

10. The method of claim 9 , further comprising inserting a cue corresponding to the specific segment within the video program at the determined time at which the specific segment ends, wherein generating the output video stream comprises replacing the cue corresponding to the specific segment with the selected advertising content corresponding to the specific segment.

11. The method of claim 1 , wherein selecting the advertising content corresponding to the specific segment comprises determining that at least a portion of the advertising content shows at least one object, person, or location which is also shown in at least a portion of the specific segment.

12. A system for dynamic advertising insertion for a user, comprising:

a memory; and

at least one processor coupled to the memory, the processor being operative:

to obtain a video program comprising a sequence of segments;

to generate metadata describing at least a specific one of the segments of the video program;

to build an enhanced user profile of the user based on the generated metadata;

based at least in part on the generated metadata describing the specific segment and the enhanced user profile built based on the generated metadata, to select advertising content corresponding to the specific segment; and

to generate an output video stream comprising the sequence of segments, wherein the selected advertising content is displayed to the user in connection with the specific segment;

wherein generating the metadata comprises at least scene detection using at least one of artificial intelligence and machine learning; and

wherein the selected advertising content is inserted, by the processor, into the output video stream between the specific segment and another segment;

wherein the at least one processor is further operative to determine, using the generated metadata, a preference of said user to watch said specific segment of the video program as opposed to the entire video program, wherein the profile is updated based at least in part on the preference and the specific segment is selected based on the preference of said user to watch said specific segment as opposed to the entire video program.

13. A computer program product comprising a non-transitory machine-readable storage medium having machine-readable program code for dynamic advertising insertion for a user embodied therewith, said machine-readable program code comprising machine-readable program code configured:

to obtain a video program comprising a sequence of segments;

to generate metadata describing at least a specific one of the segments of the video program;

to build an enhanced user profile of the user based on the generated metadata;

based at least in part on the generated metadata describing the specific segment and the enhanced user profile built based on the generated metadata, to select advertising content corresponding to the specific segment; and

to generate an output video stream comprising the sequence of segments, wherein the selected advertising content is displayed to the user in connection with the specific segment;

wherein generating the metadata comprises at least scene detection using at least one of artificial intelligence and machine learning; and

wherein the selected advertising content is inserted, by the processor, into the output video stream between the specific segment and another segment;

wherein the machine-readable program code is further configured to determine, using the generated metadata, a preference of said user to watch said specific segment of the video program as opposed to the entire video program, wherein the profile is updated based at least in part on the preference and the specific segment is selected based on the preference of said user to watch said specific segment as opposed to the entire video program.

14. The method of claim 10 , wherein the inserted cue includes the generated metadata, and wherein the metadata indicates at least one of:

at least one object, person, location, event, or action shown within at least a portion of the specific segment; and

at least one sentiment or emotion corresponding to at least a portion of the specific segment.

15. The method of claim 10 , wherein the inserted cue does not include the generated metadata, but includes an identifier permitting lookup of the metadata, and wherein the metadata indicates at least one of:

at least one object, person, location, event, or action shown within at least a portion of the specific segment; and

at least one sentiment or emotion corresponding to at least a portion of the specific segment.

16. The method of claim 10 , wherein said scene detection comprises marking said specific segment with a content taxonomy, wherein selecting advertising content corresponding to the specific segment comprises using at least one of artificial intelligence and machine learning to select said advertising content based on matching said content taxonomy.

17. The method of claim 16 , further comprising augmenting said inserted cue with said content taxonomy.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 2, 2020
From: BADAWIYEH, BASIL; PATEL, VIPUL; SCHANMAN, GARY
To: CHARTER COMMUNICATIONS OPERATING, LLC
Reel/Frame 052811/0051 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PROPERTY ASSIGNMENT PREVIOUSLY RECORDED ON REEL 051433 FRAME 0654. ASSIGNOR(S) HEREBY CONFIRMS THE SECURITY AGREEMENT. Recorded Mar 4, 2020
From: TIME WARNER CABLE ENTERPRISES, LLC; CHARTER COMMUNICATIONS OPERATING, LLC
To: WELLS FARGO TRUST COMPANY, N.A.
Reel/Frame 052091/0071 →
SECURITY INTEREST Recorded Dec 27, 2019
From: TIME WARNER CABLE ENTERPRISES, LLC; CHARTER COMMUNICATIONS OPERATING, LLC
To: WELLS FARGO TRUST COMPANY, N.A.
Reel/Frame 051433/0654 →
SECURITY INTEREST Recorded Dec 4, 2019
From: TIME WARNER CABLE ENTERPRISES, LLC; CHARTER COMMUNICATIONS OPERATING, LLC; BRIGHT HOUSE NETWORKS, LLC
To: BANK OF AMERICA, N.A.
Reel/Frame 051287/0057 →
SECURITY INTEREST Recorded Dec 4, 2019
From: CHARTER COMMUNICATIONS OPERATING, LLC; TIME WARNER CABLE ENTERPRISES, LLC; BRIGHT HOUSE NETWORKS, LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 051287/0087 →
Continuity (1)
Related Publication 20200204838A1 · Jun 25, 2020
References Cited (72)
US 7792963B2 · Gould et al. · 2010 [cited by applicant]
US 8332885B2 · Williamson et al. · 2012 [cited by applicant]
US 8819727B2 · Williamson et al. · 2014 [cited by applicant]
US 8826347B1 · Earle · 2014 [cited by applicant]
US 8869207B1 · Earle · 2014 [cited by applicant]
US 8910198B2 · Kelsen et al. · 2014 [cited by applicant]
US 8995427B1 · Barbulescu · 2015 [cited by examiner]
US 9027065B2 · Earle · 2015 [cited by applicant]
US 9124928B2 · Kelsen et al. · 2015 [cited by applicant]
US 9241179B2 · Earle · 2016 [cited by applicant]
US 9521459B2 · Earle · 2016 [cited by applicant]
US 20030056217A1 · Brooks · 2003 [cited by applicant]
US 20060130107A1 · Gonder et al. · 2006 [cited by applicant]
US 20070217436A1 · Markley et al. · 2007 [cited by applicant]
US 20070280215A1 · Gilbert et al. · 2007 [cited by applicant]
US 20080013724A1 · Shamoon · 2008 [cited by examiner]
US 20080112690A1 · Shahraray · 2008 [cited by examiner]
US 20080172293A1 · Raskin · 2008 [cited by examiner]
US 20080294577A1 · Agarwal · 2008 [cited by examiner]
US 20090248794A1 · Helms · 2009 [cited by applicant]
US 20100313236A1 · Straub · 2010 [cited by applicant]
US 20120143696A1 · Kaushik et al. · 2012 [cited by applicant]
US 20150058883A1 · Tidwell et al. · 2015 [cited by applicant]
US 20150058884A1 · Eldering et al. · 2015 [cited by applicant]
US 20150281776A1 · Desai et al. · 2015 [cited by applicant]
US 20160359937A1 · Chung · 2016 [cited by examiner]
US 20170006317A1 · LaJoie et al. · 2017 [cited by applicant]
US 20170055039A1 · Earle · 2017 [cited by applicant]
US 20190075339A1 · Smith · 2019 [cited by examiner]
Society of Cable Telecommunications Engineers (SCTE), American National Standard ANSI/SCTE 35 2017, 2017, 75 pages, https://www.scte.org/SCTEDocs/Standards/ANSI_SCTE%2035%202017.pdf. [cited by applicant]
Melissa Gallo, Interactive Advertising Bureau (IAB), Taxonomy: The Most Important Industry Initiative You've Probably Never Heard Of, Jul. 20, 2016, 3 pages, https://www.iab.com/news/taxonomy-important-industry-initiati… [cited by applicant]
Kevin Flood et al., Interactive Advertising Bureau (IAB), IAB Tech Lab Announces Final Content Taxonomy v2 Ready for Adoption, Nov. 30, 2017, 2 pages, https://iabtechlab.com/blog/iab-tech-lab-announces-final-content-tax… [cited by applicant]
Interactive Advertising Bureau (IAB), Context Taxonomy version 2.0 final release, Nov. 2017, 13 pages, http://labtechlab.com/wp-content/uploads/2017/11/IAB_Tech_Lab_Content_Taxonomy_V2_Final_2017-11.xlsx. [cited by applicant]
Interactive Advertising Bureau (IAB), Data Transparency Standards for Public Comment, May 2018, 24 pages, https://labtechlab.com/wp-content/uploads/2018/05/IAB-Tech-Lab-Data-Transparency-Framework-Public-Comment.pdf. [cited by applicant]
Interactive Advertising Bureau (IAB), Audience Taxonomy 1.0 public comment release, May 2018, 9 pages, https://labtechlab.com/wp-content/uploads/2018/05/IAB-Tech-Lab-Audience-Taxonomy-1.0-Public-Comment.xlsx. [cited by applicant]
International Press Telecommunications Council (IPTC), NewsCodes, 2018, 3 pages, https://iptc.org/standards/subject-codes/. [cited by applicant]
International Press Telecommunications Council (IPTC), NewsCodes, 2018, 3 pages, https://iptc.org/standards/newscodes/. [cited by applicant]
International Press Telecommunications Council (IPTC), NewsCodes Concept: Subject Code, 2018, 274 pages, http://cv.iptc.org/newscodes/subjectcode/. [cited by applicant]
International Press Telecommunications Council (IPTC), NewsCodes Concept: Subject Qualifier, 2018, 10 pages, http://cv.iptc.org/newscodes/subjectqualifier/. [cited by applicant]
International Press Telecommunications Council (IPTC), NewsCodes Concept: Media Topic, 2018, 237 pages, http://cv.iptc.org/newscodes/mediatopic/. [cited by applicant]
International Press Telecommunications Council (IPTC), NewsCodes Concept: Genre, 2018, 9 pages, http://cv.iptc.org/newscodes/genre/. [cited by applicant]
International Press Telecommunications Council (IPTC), NewsCodes Concept: Product Genre, 2018, 3 pages, http://cv.iptc.org/newscodes/productgenre/. [cited by applicant]
International Press Telecommunications Council (IPTC), NewsCodes Concept: Nature of a Concept, 2018, 2 pages, http://cv.iptc.org/newscodes/cpnature/. [cited by applicant]
International Press Telecommunications Council (IPTC), NewsCodes Concept: Content Warning, 2018, 2 pages, http://cv.iptc.org/newscodes/contentwarning/. [cited by applicant]
Mike Waldron, Aylien Ltd., IAB QAG Tagging is here; Semantic Targeting Made Easy, Sep. 17, 2015, 7 pages, http://blog.aylien.com/iab-qag-tagging-is-here-semantic-targeting-made/. [cited by applicant]
Will Gannon, Aylien Ltd., Which taxonomy should you use to classify news content, IAB-QAG or IPTC Subject Codes?, Nov. 12, 2018, 9 pages, http://blog.aylien.com/which-taxonomy-should-you-use-to-classify-news-content-iab… [cited by applicant]
Aylien Ltd., Text Classification, 2018, 4 pages, https://aylien.com/text-api/text-classification/. [cited by applicant]
Blanca Galego, MeaningCloud, Text Analytics & MeaningCloud 101, Feb. 14, 2017, 5 pages, https://www.meaningcloud.com/blog/text-analytics-meaningcloud-101. [cited by applicant]
Antonio Matarranz, MeaningCloud, Automatic IAB tagging enables semantic ad targeting, Jun. 13, 2016, 3 pages, https://www.meaningcloud.com/blog/automatic-iab-tagging-enables-semantic-ad-targeting. [cited by applicant]
Luca De Filippis, MeaningCloud, Updated version of the IAB model in the Deep Categorization API, Mar. 19, 2018, 2 pages, https://www.meaningcloud.com/blog/updated-iab-model-meaningcloud. [cited by applicant]
Cable Television Laboratories, Inc., CableLabs Video-On-Demand Content Specification Version 1.1, Aug. 3, 2012, 67 pages, https://specification-search.cablelabs.com/cablelabs-video-on-demand-content-specification-versio… [cited by applicant]
Cable Television Laboratories, Inc., CableLabs Content Specification Version 3.0, Nov. 4, 2015, 33 pages, https://specification-search.cablelabs.com/cablelabs-content-3-0-specification. [cited by applicant]
Cable Television Laboratories, Inc., Genre and Show Type Classification Document, Aug. 2010, 10 pages, http://mibs.cablelabs.com/namespaces/metadata/lists/genre_classification_list.pdf. [cited by applicant]
Motion Picture Laboratories, Inc., Common Metadata Version 2.7, Nov. 1, 2018, 120 pages, http://www.movielabs.com/md/md/v2.7/Common_Metadata_v2.7.pdf. [cited by applicant]
Motion Picture Laboratories, Inc., Common Media Manifest Metadata Version 1.8, Nov. 1, 2018, 107 pages, http://www.movielabs.com/md/manifest/v1.8/Manifest_v1.8.pdf. [cited by applicant]
Motion Picture Laboratories, Inc., Media Manifest Delivery Core Version 2.0, Nov. 19, 2018, 38 pages, http://www.movielabs.com/md/mmc/v2.0/Manifest_Core_v2.0.pdf. [cited by applicant]
Motion Picture Laboratories, Inc., Media Entertainment Core Metadata Version 2.6, Dec. 11, 2017, 13 pages, http://www.movielabs.com/md/mec/v2.6/Media_Ent_Core_Metadata_v2.6.pdf. [cited by applicant]
Entertainment Identifier Registry (EIDR), EIDR System Version 2.0.9 Data Fields Reference, Dec. 27, 2015, 95 pages, http://eidr.org/documents/EIDR_2.0_Data_Fields.pdf. [cited by applicant]
Society of Motion Picture and Television Engineers (SMPTE), SMPTE Standard ST 377-1:2011, Jun. 7, 2011, 183 pages, https://ieeexplore.ieee.org/document/7292073. [cited by applicant]
Society of Motion Picture and Television Engineers (SMPTE), SMPTE Engineering Guideline EG 377-3:2013, Jan. 29, 2013, 47 pages, https://ieeexplore.ieee.org/document/7291302. [cited by applicant]
Society of Motion Picture and Television Engineers (SMPTE), SMPTE Engineering Guideline EG 42:2015, Aug. 30, 2015, 26 pages, https://ieeexplore.ieee.org/document/7395464. [cited by applicant]
Society of Motion Picture and Television Engineers (SMPTE), SMPTE Standard ST 2102:2017, Oct. 23, 2017, 30 pages, https://ieeexplore.ieee.org/document/8126304. [cited by applicant]
Consumer Technology Association (CTA), Standard CTA 2033 R2016, Mar. 2008, 102 pages. [cited by applicant]
European Committee for Standardization (CEN), European Standard EN 15907:2010(E), Jul. 2010, 36 pages. [cited by applicant]
European Broadcasting Union (EBU), EBU Core Metadata Set, Tech 3293, Version 1.8, Oct. 2017, 41 pages, https://tech.ebu.ch/docs/tech/tech3293.pdf. [cited by applicant]
International Organization for Standardization (ISO), International Standard ISO 15706:2002(E), Nov. 15, 2002, 20 pages. [cited by applicant]
International Organization for Standardization (ISO), International Standard ISO/IEC 15938-5:2003(E), May 15, 2003, 738 pages. [cited by applicant]
International Organization for Standardization (ISO), International Standard ISO/IEC 21000-2:2005(E), Oct. 1, 2005, 96 pages. [cited by applicant]
International Organization for Standardization (ISO), International Standard ISO/IEC 21000-3:2003(E), Apr. 1, 2003, 38 pages. [cited by applicant]
International Organization for Standardization (ISO), International Standard ISO/IEC 21000-3:2003/Amd.1:2007(E), Feb. 1, 2007, 14 pages. [cited by applicant]
Content ID Forum (cIDF), cIDf Specification 2.0 Rev. 1.1, Apr. 1, 2007, 148 pages, http://www.npo-ba.org/cid/CIDfSpecV2R11E.pdf. [cited by applicant]
Content ID Forum (cIDF), cIDf Specification 2.0 Rev. 1.1 Appendices, Apr. 1, 2007, 75 pages, http://www.npo-ba.org/cid/cIDfSpecV2R11AE.pdf. [cited by applicant]