IP Library Granted Patent US 11,526,576
Granted Patent B2
US 11,526,576 · App. 15/809,189 · Granted Dec 13, 2022

Method and system for displaying content relating to a subject matter of a displayed media program

Inventors: Gianmarco De Francisci Morales (Catalunya, ES); Roi Blanco (Catalunya, ES); Fabrizio Silvestri (Pisa, IT)
Assignee: Pinterest, Inc.
G06F16/958H04N21/2393H04N21/23418H04N21/2665H04N21/4622
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,526,576
App. No.
15/809,189
Granted
Dec 13, 2022
Kind
B2
Abstract

Disclosed is a system and method for analyzing, by a server computer, closed captioning text associated with a media program being experienced by a user having a client device. The server computer obtains, based on the analyzing, a subject matter of a portion of the media program from the closed captioning text. The server computer constructs a query associated with the determined subject matter and submits the query to a computer network as a search query. The server computer receives, in response to the submitting of the query, content relating to the subject matter and measures an elapsed time period between the receiving of the content and the obtaining of the subject matter. If the elapsed time period is less than a predetermined period of time, the server computer communicates, to the client device, information related to the content.

Claims (63)

1. A computer-implemented method comprising:

obtaining, by a server, closed captioning information associated with a media program;

segmenting, by the server, the closed captioning information into a plurality of segments;

processing, by the server, at least one first segment from the plurality of segments to determine a subject matter associated with at least a portion of the media program;

obtaining, by the server, a content item relating to the subject matter;

determining, by the server, that an elapsed time between determination of the subject matter and obtaining of the content item is below a threshold value;

in response to the determination that the elapsed time is below the threshold value, causing, by the server, the content item to be presented on a client device;

processing, by the server, at least one second segment from the plurality of segments to determine a second subject matter associated with at least a portion of the media program;

obtaining, by the server, a second content item relating to the second subject matter;

determining, by the server, a topic change based on a comparison of the content item with the second content item; and

in response to the determination of the topic change, causing, by the server, the second content item to be presented on the client device.

2. The computer-implemented method of claim 1 , wherein processing the at least one first segment from the plurality of segments to determine the subject matter includes identifying a topic corresponding to the at least one first segment.

3. The computer-implemented method of claim 1 , wherein obtaining the closed captioning information associated with the media program includes:

determining an identity of the media program based at least in part on a fingerprint associated with the media program; and

obtaining the closed captioning information based at least in part on the identity of the media program.

4. The computer-implemented method of claim 1 , wherein the content item includes a plurality of content items and the method further comprises:

prior to causing the content item to be presented on the client device, generating a ranking associated with the plurality of content items; and

causing the plurality of content items to be presented on the client device in accordance with the ranking.

5. The computer-implemented method of claim 1 , further comprising:

determining, by the server, that a second elapsed time between determination of the second subject matter and obtaining of the second content item is below the threshold value, and

wherein causing the second content item to be presented on the client device is further based on the determination that the second elapsed time is below the threshold value.

6. A computing system, comprising:

one or more processors; and

a memory coupled to the one or more processors and storing program instructions that when executed by the one or more processors, cause the one or more processors to at least:

obtain closed captioning information associated with a media program;

segment the closed captioning information into a plurality of segments;

process at least one segment of the plurality of segments to determine a subject matter associated with a first portion of the media program;

obtain a content relating to the subject matter;

cause the content to be presented on a client device;

process at least one second segment from the plurality of segments to determine a second subject matter associated with at least a portion of the media program;

obtain a second content relating to the second subject matter;

determine a topic change based on a comparison of the content with the second content; and

in response to the determination of the topic change, cause the second content to be presented on the client device.

7. The computing system of claim 6 , wherein processing of the at least one segment of the plurality of segments to determine the subject matter includes identifying a topic corresponding to the at least one first segment using a sliding window scheme.

8. The computing system of claim 6 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to at least:

determine that an elapsed time between determination of the subject matter and obtaining of the content is below a threshold value,

wherein the content is caused to be presented on the client device in response to the determination that the elapsed time is below the threshold value.

9. The computing system of claim 6 , wherein;

an identity associated with the media program is determined based at least in part on an audio fingerprint associated with the media program; and

obtaining the closed captioning information associated with the media program is based at least in part on the identity associated with the media program.

10. The computing system of claim 6 , wherein the content includes a plurality of content and the instructions, when executed by the one or more processors, further cause the one or more processors to at least:

prior to causing the content to be presented on the client device, generate a ranking associated with the plurality of content; and

cause the plurality of content to be presented on the client device in accordance with the ranking.

11. The computing system of claim 6 , wherein each of the plurality of segments includes a respective timestamp indicating a time at which the segment aired.

12. The computing system of claim 8 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to at least:

determine that a second elapsed time between determination of the second subject matter and obtaining of the second content is below the threshold value, and

wherein causing the second content to be presented on the client device is further based on the determination that the second elapsed time is below the threshold value.

13. The computing system of claim 12 , wherein the closed captioning information is continuously obtained such that the content and the second content are presented on the client device as a continuous stream as the closed captioning information is obtained.

14. The computing system of claim 13 , wherein each of the plurality of segments includes a monotonically increasing timestamp indicating a time at which the segment was aired, such that the continuous stream can be reproduced based at least in part on the monotonically increasing timestamps.

15. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to at least:

obtain closed captioning information associated with a media program;

segment the closed captioning information into a plurality of segments;

process at least one first segment from the plurality of segments to determine a subject matter associated with at least a portion of the media program;

obtain a content item relating to the subject matter;

determine that an elapsed time between determination of the subject matter and obtaining of the content item is below a threshold value;

in response to the determination that the elapsed time is below the threshold value, cause the content item to be presented on a client device;

process at least one second segment from the plurality of segments to determine a second subject matter associated with at least a portion of the media program;

obtain a second content item relating to the second subject matter;

determine a topic change based on a comparison of the content item with the second content item; and

in response to the determination of the topic change, cause the second content item to be presented on the client device.

16. The non-transitory computer-readable storage medium of claim 15 , wherein each of the plurality of segments includes a respective timestamp indicating a time at which the segment aired.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the closed captioning information is continuously obtained such that the content item and the second content item are presented on the client device as a continuous stream as the closed captioning information is obtained.

18. The non-transitory computer-readable storage medium of claim 17 , wherein each of the plurality of segments includes a monotonically increasing timestamp indicating a time at which the segment was aired such that the continuous stream can be reproduced based at least in part on the monotonically increasing timestamps.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 5, 2019
From: EXCALIBUR IP, LLC
To: PINTEREST, INC.
Reel/Frame 049964/0350 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 5, 2018
From: DE FRANCISCI MORALES, GIANMARCO; BLANCO, ROI; SILVESTRI, FABRIZIO
To: YAHOO! INC.
Reel/Frame 044544/0708 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 5, 2018
From: YAHOO! INC.
To: EXCALIBUR IP, LLC
Reel/Frame 045010/0001 →
Continuity (2)
Continuation 13891854 · May 10, 2013
Related Publication 20180095980A1 · Apr 5, 2018