IP Library Granted Patent US 11,671,675
Granted Patent B2
US 11,671,675 · App. 17/092,858 · Granted Jun 6, 2023

Augmentation of audio/video content with enhanced interactive content

Inventors: Jing Sun (King of Prussia, PA); Yunbo Tang (Malvern, PA)
Assignee: Rovi Guides, Inc.
H04N21/4888H04N21/4884H04N21/8133H04N21/8456
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,671,675
App. No.
17/092,858
Granted
Jun 6, 2023
Kind
B2
Abstract

Provided are systems and methods for augmenting audio/video content with enhanced interactive content. Events are detected in the audio/video content and contextual information is determined corresponding to the events using enhanced metadata and content-specific data. Indicators are displayed to indicate the occurrence of an event and the information about the event is provided.

Claims (55)

1. A computer-implemented method for augmenting audio/video content, the method comprising:

determining, using processing equipment, that the audio/video content is related to a sports broadcast of a particular sport;

identifying, using the processing equipment, enhanced metadata associated with the particular sport in an enhanced metadata database;

identifying, using the processing equipment, a plurality of possible events that may occur during the particular sport and a rule associated with each of the possible events, based on the enhanced metadata associated with the particular sport;

detecting, using the processing equipment, an event and a corresponding rule among the plurality of possible events and rules associated with each of the possible events during a time segment of the audio/video content based on data in the audio/video content corresponding to the time segment;

identifying, using the processing equipment, enhanced metadata associated with the rule corresponding to the event in the enhanced metadata database;

extracting, using the processing equipment, content-specific data from the audio/video content;

augmenting, using the processing equipment, the audio/video content with the identified enhanced metadata associated with the rule corresponding to the event and with the content-specific data to generate augmented content; and

providing the augmented content for display on user equipment during playback of the time segment.

2. The method of claim 1 , wherein the plurality of possible events that may occur during the particular sport comprise a list of possible events.

3. The method of claim 1 , wherein the time segment is a first time segment, the enhanced metadata associated with the rule corresponding to the event is a first enhanced metadata, and

wherein the method further comprises:

identifying a second event and a second corresponding rule among the plurality of possible events and rules associated with each of the possible events in a second time segment of the audio/video content that at least partially overlaps with the first time segment; and

identifying second enhanced metadata associated with the second rule corresponding to the second event, wherein augmenting the audio/video content comprises augmenting the audio/video content with the identified first enhanced metadata associated with the rule corresponding to the event, the content-specific data, and the identified second enhanced metadata associated with the second rule corresponding to the second event to generate augmented data.

4. The method of claim 3 , further comprising extracting second content-specific data from the audio/video content, wherein augmenting the audio/video content comprises augmenting the audio/video content with the identified first enhanced metadata, the content-specific data, the identified second enhanced metadata associated with the second rule corresponding to the second event, and the second content-specific data to generate the augmented data.

5. The method of claim 1 , wherein providing the augmented content for display on the user equipment during playback of the time segment comprises causing to be simultaneously displayed on the user equipment during the time segment content from the audio/video content and interactive content associated with the event.

6. The method of claim 1 , wherein extracting the content-specific data from the audio/video content comprises extracting the content-specific data from closed-captioning data of the audio/video content.

7. A system for augmenting audio/video content, the system comprising:

electronic storage equipment configured to store an enhanced metadata database; and

processing equipment coupled to the electronic storage equipment and configured to:

determine that the audio/video content is related to a sports broadcast of a particular sport;

identify enhanced metadata associated with the particular sport in an enhanced metadata database;

identify a plurality of possible events that may occur during the particular sport and a rule associated with each of the possible events, based on the enhanced metadata associated with the particular sport;

detect an event and a corresponding rule among the plurality of possible events and rules associated with each of the possible events during a time segment of the audio/video content based on data in the audio/video content corresponding to the time segment;

identify enhanced metadata associated with the rule corresponding to the event in the enhanced metadata database;

extract content-specific data from the audio/video content;

augment the audio/video content with the identified enhanced metadata associated with the rule corresponding to the event and with the content-specific data to generate augmented content; and

provide the augmented content for display on user equipment during playback of the time segment.

8. The system of claim 7 , wherein the plurality of possible events that may occur during the particular sport comprise a list of possible events.

9. The system of claim 7 , wherein the time segment is a first time segment, the enhanced metadata associated with the rule corresponding to the event is a first enhanced metadata, and

wherein the processing equipment is further configured to:

identify a second event and a second corresponding rule among the plurality of possible events and rules associated with each of the possible events in a second time segment of the audio/video content that at least partially overlaps with the first time segment; and

identify second enhanced metadata associated with the second rule corresponding to the second event, wherein augmenting the audio/video content comprises augmenting the audio/video content with the identified first enhanced metadata associated with the rule corresponding to the event, the content-specific data, and the identified second enhanced metadata associated with the second rule corresponding to the second event to generate augmented data.

10. The system of claim 9 , wherein the processing equipment is further configured to:

extract second content-specific data from the audio/video content; and

augment the audio/video content with the identified first enhanced metadata, the content-specific data, the identified second enhanced metadata associated with the second rule corresponding to the second event, and the second content-specific data to generate the augmented data.

11. The system of claim 7 , wherein the processing equipment is further configured to cause to be simultaneously displayed on the user equipment during the time segment content from the audio/video content and interactive content associated with the event.

12. The system of claim 7 , wherein the processing equipment is further configured to extract the content-specific data from closed-captioning data of the audio/video content.

13. A non-transitory computer readable medium having computer instructions stored thereon for programming a computer to perform a method comprising:

determining, using processing equipment, that audio/video content is related to a sports broadcast of a particular sport;

identifying, using the processing equipment, enhanced metadata associated with the particular sport in an enhanced metadata database;

identifying, using the processing equipment, a plurality of possible events that may occur during the particular sport and a rule associated with each of the possible events, based on the enhanced metadata associated with the particular sport;

detecting, using the processing equipment, an event and a corresponding rule among the plurality of possible events and rules associated with each of the possible events during a time segment of the audio/video content based on data in the audio/video content corresponding to the time segment;

identifying, using the processing equipment, enhanced metadata associated with the rule corresponding to the event in the enhanced metadata database;

extracting, using the processing equipment, content-specific data from the audio/video content;

augmenting, using the processing equipment, the audio/video content with the identified enhanced metadata associated with the rule corresponding to the event and with the content-specific data to generate augmented content; and

providing the augmented content for display on user equipment during playback of the time segment.

14. The non-transitory computer readable medium of claim 13 , wherein the plurality of possible events that may occur during the particular sport comprise a list of possible events.

15. The non-transitory computer readable medium of claim 13 , wherein the time segment is a first time segment the enhanced metadata associated with the rule corresponding to the event is a first enhanced metadata, and

wherein the method further comprises:

identifying a second event and a second corresponding rule among the plurality of possible events and rules associated with each of the possible events in a second time segment of the audio/video content that at least partially overlaps with the first time segment; and

identifying second enhanced metadata associated with the second rule corresponding to the second event, wherein augmenting the audio/video content comprises augmenting the audio/video content with the identified first enhanced metadata associated with the rule corresponding to the event, the content-specific data, and the identified second enhanced metadata associated with the second rule corresponding to the second event to generate augmented data.

16. The non-transitory computer readable medium of claim 15 , wherein the method further comprises extracting second content-specific data from the audio/video content, wherein augmenting the audio/video content comprises augmenting the audio/video content with the identified first enhanced metadata, the content-specific data, the identified second enhanced metadata associated with the second rule corresponding to the second event, and the second content-specific data to generate the augmented data.

17. The non-transitory computer readable medium of claim 13 , wherein providing the augmented content for display on the user equipment during playback of the time segment comprises causing to be simultaneously displayed on the user equipment during the time segment content from the audio/video content and interactive content associated with the event.

18. The non-transitory computer readable medium of claim 13 , wherein extracting the content-specific data from the audio/video content comprises extracting the content-specific data from closed-captioning data of the audio/video content.

Assignments (3)
CHANGE OF NAME Recorded Oct 3, 2024
From: ROVI GUIDES, INC.
To: ADEIA GUIDES INC.
Reel/Frame 069106/0171 →
SECURITY INTEREST Recorded May 3, 2023
From: ADEIA GUIDES INC.; ADEIA IMAGING LLC; ADEIA MEDIA HOLDINGS LLC; ADEIA MEDIA SOLUTIONS INC.; ADEIA SEMICONDUCTOR ADVANCED TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR BONDING TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR INC.; ADEIA SEMICONDUCTOR SOLUTIONS LLC; ADEIA SEMICONDUCTOR TECHNOLOGIES LLC; ADEIA SOLUTIONS LLC
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 063529/0272 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 7, 2021
From: SUN, JING; TANG, YUNBO
To: ROVI GUIDES, INC.
Reel/Frame 054842/0528 →
Continuity (2)
Continuation 16144340 · Sep 27, 2018
Related Publication 20210127174A1 · Apr 29, 2021
Cited By (2)
US 12,293,759 US 12,316,923