IP Library Granted Patent US 9,942,599
Granted Patent B2
US 9,942,599 · App. 15/442,018 · Granted Apr 10, 2018

Methods and apparatus to synchronize second screen content with audio/video programming using closed captioning data

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,942,599
App. No.
15/442,018
Granted
Apr 10, 2018
Kind
B2
Abstract

A method for synchronized utilization of an electronic device is provided. The method receives closed captioning data from an audio/video content receiver for a set of audio/video content; retrieves detail for a first event occurring in the set of audio/video content, wherein the first event is indicated by the received closed captioning data; and presents content to a user, using the electronic device, based on the retrieved detail.

Claims (90)

1. A method for synchronized utilization of an electronic device, the method comprising:

receiving closed captioning data from an audio/video content receiver for a set of audio/video content;

retrieving detail for a first event occurring in the set of audio/video content, wherein the first event is indicated by the received closed captioning data, by:

accessing a remote server to retrieve a playlist comprising a list of events occurring in the set of audio/video content and detail associated with each of the list of events;

wherein the detail associated with each of the events comprises a closed captioning reference string, a time offset, and a reference to second screen content for display after occurrence of the closed captioning reference string and passage of the time offset in the set of audio/video content;

performing a lookup to compare the closed captioning data to a plurality of closed captioning reference strings cataloged in the playlist, to locate a match, wherein the plurality of closed captioning reference strings comprises the closed captioning reference string; and

when a match is located,

identifying appropriate second screen content cataloged in the playlist;

identifying a time offset from the first event, wherein the time offset is associated with the first event and the appropriate second screen content in the playlist; and

presenting the appropriate second screen content to a user, after the time offset from the first event during playback of the set of audio/video content, by the electronic device, based on the retrieved detail, wherein the appropriate second screen content comprises the second screen content.

2. The method of claim 1 , wherein the closed captioning data comprises a proper noun, and wherein the first event is associated with occurrence of the proper noun in the closed captioning data of the set of audio/video content.

3. The method of claim 2 , further comprising:

identifying the proper noun, based on the occurrence of the proper noun;

determining a genre for the set of audio/video content;

determining a context for the proper noun based on the determined genre of the set of audio/video content; and

presenting the content to the user based on the retrieved detail and the determined context.

4. The method of claim 2 , further comprising:

identifying the proper noun, based on the occurrence of the proper noun;

evaluating a group of words surrounding the proper noun;

determining a context for the proper noun based on the evaluated group of words; and

presenting the content to the user based on the retrieved detail and the determined context.

5. The method of claim 1 , wherein the presenting step further comprises:

executing a command to open a web browser, via the electronic device; and

presenting a website to the user, using the opened web browser, wherein the retrieved detail comprises a uniform resource locator (URL), and wherein the content comprises the web site.

6. The method of claim 1 , wherein the presenting step further comprises:

presenting a plurality of selectable options to the user, via the electronic device, wherein the detail comprises the plurality of selectable options; and

receiving user input to select one of the plurality of selectable options;

wherein the content comprises the selected one of the plurality of selectable options.

7. A system for processing audio/video content, the system comprising:

an audio/video content receiver, configured to provide audio/video programming to a display device and to provide closed captioning data;

a second screen electronic device, configured for use in conjunction with viewing the audio/video programming on the display device, wherein the second screen electronic device is configured to:

receive the closed captioning data provided by the audio/video content receiver;

access a remote server to retrieve detail for an event occurring in the audio/video programming, by:

accessing the remote server to retrieve a playlist comprising a list of events occurring in the set of audio/video content and detail associated with each of the list of events;

wherein the detail for the event comprises a closed captioning reference string, a time offset, and a reference to second screen content for display after occurrence of the closed captioning reference string and passage of the time offset in the set of audio/video content; and

wherein the event is indicated by the closed captioning data;

perform a lookup to compare the closed captioning data to a plurality of closed captioning reference strings cataloged in the playlist, to locate a match, wherein the plurality of closed captioning reference strings comprises the closed captioning reference string; and

when a match is located,

identify appropriate second screen content cataloged in the playlist;

identify a time offset from the first event, wherein the time offset is associated with the first event and the appropriate second screen content in the playlist; and

present the appropriate second screen content to a user, after the time offset from the first event during playback of the set of audio/video content, based on the retrieved detail, wherein the appropriate second screen content comprises the second screen content.

8. The system of claim 7 , wherein the audio/video content receiver comprises a set-top box (STB) configured to communicate with the second screen electronic device over a wireless local area network (WLAN).

9. The system of claim 7 , further comprising a single computing device,

wherein the single computing device comprises the audio/video content receiver and the second screen electronic device; and

wherein the audio/video programming and the content are presented using a shared screen of the single computing device.

10. The system of claim 7 , further comprising:

a playlist generator, configured to:

manually designate the event occurring in the audio/video programming; and

associate the event with a closed captioning reference string, wherein the closed captioning reference string comprises a subset of the closed captioning data; and

a playlist creation module, configured to:

further associate the manually designated event with the content for presentation via the second screen electronic device; and

generate the playlist cataloging the event, the closed captioning reference string, and the content;

wherein the detail comprises the playlist.

11. The system of claim 7 , wherein the retrieved detail comprises at least the event, an associated closed captioning reference string, and associated content detail.

12. The system of claim 11 , wherein the content comprises a website, and wherein the associated content detail comprises a uniform resource locator (URL).

13. The system of claim 11 , wherein the content comprises text displayed via the second screen electronic device, and wherein the associated content detail comprises a representation of the text.

14. A computing device configured for use in conjunction with an audio/video content receiver that provides audio/video programming and closed captioning data to a display device, the computing device comprising:

a system memory element;

a wireless communication module, configured to communicate data between the computing device and one or more remote servers within a wireless range of the computing device;

a user interface comprising a display for presenting second screen content;

at least one processor, communicatively coupled to the system memory element, the wireless communication module, and the user interface, the at least one processor configured to:

receive, via the wireless communication module, the closed captioning data provided by the audio/video content receiver;

access, via the wireless communication module, a remote server to retrieve detail for an event occurring in the audio/video programming, by:

accessing the remote server to retrieve a playlist comprising a list of events occurring in the set of audio/video content and detail associated with each of the list of events;

wherein the detail for the event comprises a closed captioning reference string, a time offset, and a reference to second screen content for display after occurrence of the closed captioning reference string and passage of the time offset in the set of audio/video content; and

wherein the event is indicated by the closed captioning data;

perform a lookup to compare the closed captioning data to a plurality of closed captioning reference strings cataloged in the playlist, to locate a match, wherein the plurality of closed captioning reference strings comprises the closed captioning reference string; and

when a match is located,

identify appropriate second screen content cataloged in the playlist;

identify a time offset from the first event, wherein the time offset is associated with the first event and the appropriate second screen content in the playlist; and

present, via the user interface, the appropriate second screen content to a user, after the time offset from the first event during playback of the set of audio/video content, based on the retrieved detail, wherein the appropriate second screen content comprises the second screen content.

15. The computing device of claim 14 , wherein the closed captioning data comprises a proper noun, and wherein the first event is associated with occurrence of the proper noun in the closed captioning data of the set of audio/video content.

16. The computing device of claim 15 , wherein the at least one processor is further configured to:

identify the proper noun, based on the occurrence of the proper noun;

determine a genre for the set of audio/video content;

determine a context for the proper noun based on the determined genre of the set of audio/video content; and

present the content to the user based on the retrieved detail and the determined context.

17. The computing device of claim 15 , wherein the at least one processor is further configured to:

identify the proper noun, based on the occurrence of the proper noun;

evaluate a group of words surrounding the proper noun;

determine a context for the proper noun based on the evaluated group of words; and

present the content to the user based on the retrieved detail and the determined context.

18. The computing device of claim 14 , wherein the at least one processor is configured to present, via the user interface, the appropriate second screen content, by:

executing a command to open a web browser, via the electronic device; and

presenting a website to the user, using the opened web browser, wherein the retrieved detail comprises a uniform resource locator (URL), and wherein the content comprises the website.

19. The computing device of claim 14 , wherein the at least one processor is configured to present, via the user interface, the appropriate second screen content, by:

presenting a plurality of selectable options to the user, via the electronic device, wherein the detail comprises the plurality of selectable options; and

receiving user input to select one of the plurality of selectable options;

wherein the content comprises the selected one of the plurality of selectable options.

20. The computing device of claim 14 , wherein the retrieved detail comprises at least the event, an associated closed captioning reference string, and associated content detail comprising at least a uniform resource locator (URL).

Assignments (4)
SECURITY INTEREST Recorded Nov 30, 2021
From: DISH BROADCASTING CORPORATION; DISH NETWORK L.L.C.; DISH TECHNOLOGIES L.L.C.
To: U.S. BANK, NATIONAL ASSOCIATION, AS COLLATERAL AGENT
Reel/Frame 058295/0293 →
CONVERSION Recorded Aug 7, 2018
From: ECHOSTAR TECHNOLOGIES L.L.C.
To: DISH TECHNOLOGIES L.L.C.
Reel/Frame 046737/0610 →
CHANGE OF NAME Recorded May 3, 2018
From: ECHOSTAR TECHNOLOGIES L.L.C.
To: DISH TECHNOLOGIES L.L.C.
Reel/Frame 046678/0224 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 24, 2017
From: CASAGRANDE, STEVEN MICHAEL
To: ECHOSTAR TECHNOLOGIES L.L.C.
Reel/Frame 041372/0962 →