IP Library Granted Patent US 9,542,489
Granted Patent B2
US 9,542,489 · App. 12/838,405 · Granted Jan 10, 2017

Estimating social interest in time-based media

Inventors: Michael Ben Fleischman (Somerville, MA); Deb Kumar Roy (Arlington, MA)
Assignee: Bluefin Labs, Inc.
G06F17/3082G06F3/048G06F3/0482G06F3/0484G06F17/3053G06F17/30572G06F17/30867G06Q30/02H04L51/32H04N21/4668H04N21/4788H04N21/4826H04N21/812H04N21/8456
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,542,489
App. No.
12/838,405
Granted
Jan 10, 2017
Kind
B2
Abstract

Social media content items are mapped to relevant time-based media events. These mappings may be used as the basis for multiple applications, such as ranking of search results for time-based media, automatic recommendations for time-based media, prediction of audience interest for media purchasing/planning, and estimating social interest in the time-based media. Social interest in time-based media (e.g., video and audio streams and recordings) segments is estimated through a process of data ingestion and integration. The estimation process determines social interest in specific events represented as segments in time-based media, such as particular plays in a sporting event, scenes in a television show, or advertisements in an advertising block. The resulting estimates of social interest also can be graphically displayed.

Claims (85)

1. A computer-executed method for associating social media content items with a time-based media event, the method comprising:

accessing from a social networking system a plurality of candidate social media content items authored by users of the social networking system;

for each of the candidate social media content items, determining a confidence score indicative of a probability that the candidate social media content item is relevant to the event;

aligning with the event, based on their respective confidence scores, a subset of the plurality of the social media content items; and

collecting in a data store the alignments between the event and the subset of the plurality of the social media content items.

2. The computer-executed method of claim 1 , further comprising segmenting a received time-based medium into a plurality of segments corresponding to time-based media events.

3. The computer-executed method of claim 1 , wherein determining the confidence score indicative of the probability that the candidate social media content item is relevant to the event further comprises:

extracting event features from annotations associated with the event;

extracting social media features from the plurality of social media content items; and

mapping the event to the social media content items based on a relationship between the event features and social media features.

4. The computer-executed method of claim 3 , further comprising annotating the event with the annotations using metadata instances relevant to the event.

5. The computer-executed method of claim 1 , further comprising:

aggregating the confidence scores of the associated subset of candidate social media content items to produce an aggregate score; and

determining a level of social interest in the event based upon the aggregate score.

6. A computer-executed method for determining social interest in an event in a time-based media, the method comprising:

accessing from a social networking system a plurality of candidate social media content items authored by users of the social networking system;

for each of the candidate social media content items, determining a confidence score indicative of a probability that the candidate social media content item is relevant to the event;

aligning with the event, based on their respective confidence scores, a subset of the plurality of the social media content items;

aggregating the confidence scores of the aligned subset of candidate social media content items to produce an aggregate score; and

determining a level of social interest in the event based upon the aggregate score.

7. The computer-executed method of claim 6 , wherein determining the confidence score indicative of the probability that the candidate social media content item is relevant to the event further comprises:

extracting event features from annotations associated with the event;

extracting social media features from the plurality of social media content items; and

mapping the event to the social media content items based on a relationship between the event features and social media features.

8. The computer-executed method of claim 7 , further comprising annotating the event with the annotations using metadata instances relevant to the event.

9. A computer-executed method for determining social interest in time-based media events, the method comprising:

selecting a plurality of events in a time-based medium;

accessing from a social networking system a plurality of candidate social media content items authored by users of the social networking system that are potentially relevant to the events in the time based medium;

for each social media content item, determining a confidence score for the social media content item with respect to at least one of the plurality of events, the confidence score indicating a probability that the social media content item is relevant to the at least one event; and

for each of the plurality of events:

aligning with the event, based on their respective confidence scores, a subset of the plurality of the social media content items;

aggregating the confidence scores of the social media content items having a confidence score for the event; and

determining a level of social interest for in the event based on the aggregate score.

10. The computer-executed method of claim 9 , further comprising collecting in a data store the alignments between the event and the subset of the plurality of the social media content items.

11. The computer-executed method of claim 9 , wherein for a selected event, determining the confidence score indicative of the probability that the candidate social media content item is relevant to the event further comprises:

extracting event features from annotations associated with the event;

extracting social media features from the plurality of social media content items; and

mapping the event to the social media content items based on a relationship between the event features and social media features.

12. The computer-executed method of claim 11 , further comprising annotating the event with the annotations using metadata instances relevant to the event.

13. A computer-executed method for mapping social content items to time-based media events, comprising:

accessing from a social networking system a plurality of social media content items authored by users of the social networking system;

aligning a plurality of metadata instances to segments of time-based media corresponding to events in the time-based media to form annotated events; and

mapping to the annotated events the social media content items relevant to the annotated events.

14. The computer-executed method of claim 13 , wherein aligning a plurality of metadata instances to segments of time-based media corresponding to events in the time-based media to form annotated events further comprises:

receiving the plurality of metadata instances;

segmenting the time-based media into the segments corresponding to events in the time-based media, each segment having a beginning and an end; and

determining, for each metadata instance, a segment of the time-based media that most likely aligns with the metadata instance.

15. The computer-executed method of claim 13 , wherein mapping to the annotated events a plurality of social media content items relevant to the annotated events further comprises:

extracting social media features from the plurality of social media content items;

extracting event features from the annotated events; and

mapping the annotated events to the plurality of social media content items based on a relationship between the event features and social media features.

16. A system for associating social media content items with a time-based media event, the system comprising:

means for accessing from a social networking system a plurality of candidate social media content items authored by users of the social networking system;

means for determining a confidence score for each of the candidate social media content items indicative of a probability that the candidate social media content item is relevant to the event;

means for aligning with the event, based on their respective confidence scores, a subset of the plurality of the social media content items; and

a data store for collecting the alignments between the event and the subset of the plurality of the social media content items.

17. The system of claim 16 , further comprising:

means for extracting event features from annotations associated with the event;

means for extracting social media features from the plurality of social media content items; and

means for mapping the event to the social media content items based on a relationship between the event features and social media features.

18. The system of claim 17 , further comprising annotating the event with the annotations using metadata instances relevant to the event.

19. The system of claim 16 , further comprising:

means for aggregating the confidence scores of the associated subset of candidate social media content items to produce an aggregate score; and

means for determining a level of social interest in the event based upon the aggregate score.

20. A system for mapping social content items to time-based media events, comprising:

means for accessing from a social networking system a plurality of social media content items authored by users of the social networking system;

means for aligning a plurality of metadata instances to segments of time-based media corresponding to events in the time-based media to form annotated events; and

means for mapping to the annotated events the social media content items relevant to the annotated events.

21. A system for associating social media content items with a time-based media event, the system comprising:

a computer processor; and

a computer-readable storage medium storing computer program modules configured to execute on the computer processor, the computer program modules comprising:

a data ingestion engine configured to access from a social networking system a plurality of candidate social media content items authored by users of the social networking system;

a media/event alignment engine configured to determine a confidence score for each of the candidate social media content items indicative of a probability that the candidate social media content item is relevant to the event and to align with the event, based on their respective confidence scores, a subset of the plurality of the social media content items; and

a data store for collecting the alignments between the event and the subset of the plurality of the social media content items.

22. The system of claim 21 , the computer program modules further comprising:

a comparative feature extraction engine configured to extract event features from annotations associated with the event and to extract social media features from the plurality of social media content items; and

the media/event alignment engine further configured to map the event to the social media content items based on a relationship between the event features and social media features.

23. The system of claim 22 , the computer program modules further comprising:

an annotation engine configured to annotate the event with the annotations using metadata instances relevant to the event.

24. The system of claim 21 , the computer program modules further comprising:

a social interest estimator configured to aggregate the confidence scores of the associated subset of candidate social media content items to produce an aggregate score and to determine a level of social interest in the event based upon the aggregate score.

25. A computer system for mapping social content items to time-based media events, the computer system comprising a processor and a memory and further comprising:

a data ingestion engine, executed by the computer system, configured to access from a social networking system a plurality of candidate social media content items authored by users of the social networking system;

a metadata alignment engine, executed by the computer system, configured to align a plurality of metadata instances to segments of time-based media corresponding to events in the time-based media to form annotated events; and

a media/event alignment engine, executed by the computer system, configured to map to the annotated events a plurality of social media content items relevant to the annotated events.

Assignments (8)
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENT RIGHTS (REEL 062079, FRAME 0677) Recorded Mar 3, 2026
From: MORGAN STANLEY SENIOR FUNDING, INC., AS COLLATERAL AGENT
To: X CORP. (F/K/A TWITTER, INC.)
Reel/Frame 075015/0574 →
RELEASE OF SECURITY INTEREST Recorded Apr 30, 2025
From: MORGAN STANLEY SENIOR FUNDING, INC., AS COLLATERAL AGENT
To: X CORP. (F/K/A TWITTER, INC.)
Reel/Frame 071127/0240 →
RELEASE OF SECURITY INTEREST Recorded Mar 27, 2025
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: X CORP. (F/K/A TWITTER, INC.)
Reel/Frame 070670/0857 →
SECURITY INTEREST Recorded Oct 28, 2022
From: TWITTER, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 062079/0677 →
SECURITY INTEREST Recorded Oct 28, 2022
From: TWITTER, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 061804/0001 →
SECURITY INTEREST Recorded Oct 28, 2022
From: TWITTER, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 061804/0086 →
CHANGE OF NAME Recorded May 16, 2011
From: BLUEFIN LAB, INC.
To: BLUEFIN LABS, INC.
Reel/Frame 026285/0171 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 15, 2010
From: FLEISCHMAN, MICHAEL BEN; ROY, DEB KUMAR
To: BLUEFIN LAB, INC.
Reel/Frame 025143/0015 →
Continuity (2)
Provisional Application 61226002 · Jul 16, 2009
Related Publication 20110040760A1 · Feb 17, 2011