IP Library Granted Patent US 8,572,088
Granted Patent B2
US 8,572,088 · App. 11/256,411 · Granted Oct 29, 2013

Automated rich presentation of a semantic topic

Inventors: Lie Lu (Beijing, CN); Wei-Ying Ma (Beijing, CN); Zhiwei Li (Beijing, CN)
Assignee: Microsoft Corporation
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,572,088
App. No.
11/256,411
Granted
Oct 29, 2013
Kind
B2
Abstract

Automated rich presentation of a semantic topic is described. In one aspect, respective portions of multimodal information corresponding to a semantic topic are evaluated to locate events associated with the semantic topic. The probability that a document belongs to an event is determined based on document inclusion of one or more of persons, times, locations, and keywords, and document distribution along a timeline associated with the event. For each event, one or more documents objectively determined to be substantially representative of the event are identified. One or more other types of media (e.g., video, images, etc.) related to the event are then extracted from the multimodal information. The representative documents and the other media are for presentation to a user in a storyboard.

Claims (49)

1. A computer-implemented method comprising:

determining a semantic topic;

evaluating respective portions of multimodal information corresponding to the semantic topic to identify events, each event being associated with one or more of person, time, location, and keyword;

for each document in the respective portion, calculating probability that the document belongs to an event of the events based on a generative model and document distribution along a timeline associated with the event;

for each event in least a subset of the events:

objectively identifying one or more representative documents that are of greater relevance to the event as compared to other documents;

extracting other media corresponding to the representative documents from the multimodal information, the representative documents and the other media being objectively most representative of the semantic topic; and

wherein the one or more representative documents and the other media are for presentation to a user in a storyboard.

2. The method of claim 1 , wherein the at least a subset of the events is an event summary.

3. The method of claim 1 , wherein calculating the probability further comprises:

identifying a salient number of events corresponding to the semantic topic, the salient number of events being less than a total number of the events; and

calculating the probability using the salient number of events.

4. The method of claim 1 , wherein calculating the probability further comprises representing the probability associated with time as a function of where a date of the document lies with respect to event duration.

5. The method of claim 1 , wherein calculating the probability further comprises:

independently estimating, for respective person, location, and keyword models, model parameters by iterative expectation and maximization operations; and

calculating the probability using the models in view of temporal continuity of the event and any overlap of two or more of the events.

6. The method of claim 1 , further comprising, for each document in the representative documents, removing any resource associated with an advertisement provided an entity other than an entity associated with a web site from which the document was obtained.

7. The method of claim 1 , further comprising, for each event in the at least a subset of the events, if the representative documents include multiple documents, removing any duplicate image from the multiple documents.

8. The method of claim 1 , wherein the one or more representative documents and the other media are representative content, and wherein the method further comprises integrating the representative content into the storyboard for presentation to a user, the storyboard providing a concise overview of salient event(s) and associated multimodal information regarding the semantic topic.

9. The method of claim 1 , further comprising synchronizing the representative content of the storyboard layout with music.

10. The method of claim 9 , wherein the synchronizing further comprises:

identifying music sub-clips and a corresponding timeline; and

synchronizing event slide transitions and the storyboard layout with the music sub-clips using the timeline.

11. The method of claim 9 , wherein the synchronizing further comprises:

identifying music sub-clips and a corresponding timeline;

synchronizing event slide transitions and the storyboard layout with the music sub-clips using the timeline; and

wherein a length of an event slide is equal to a corresponding length of a respective music sub-clip.

12. The method of claim 9 , wherein the synchronizing further comprises:

identifying music sub-clips and a corresponding timeline;

synchronizing event slide transitions and the storyboard layout with the music sub-clips using the timeline; and

wherein each event associated with the event slide transitions is objectively determined to belong to a set of events in the at least a subset of events that are more important than other events in the at least a subset of events.

13. A computer implemented method comprising:

determining a semantic topic;

extracting, from multimedia data, multimodal information relevant to the semantic topic;

evaluating respective portions of the multimodal information to identify events, each event being associated with one or more of person, time, location, and keyword;

for each document in the respective portion, calculating probability that the document belongs to an event of the events based on a generative model and document distribution along a timeline associated with the event;

generating an event summary summarizing the events;

for each event in the event summary:

objectively identifying one or more representative documents that are of greater relevance to the event as compared to other documents; and

extracting other media corresponding to the representative documents from the multimodal information, the representative documents and the other media being representative content;

integrating the representative content into a storyboard layout for presentation to a user, the storyboard layout providing a concise overview of salient event(s) and associated multimodal information regarding the semantic topic.

14. The method of claim 13 , wherein calculating the probability further comprises:

identifying a salient number of events corresponding to the semantic topic, the salient number of events being less than a total number of the events; and

calculating the probability using the salient number of events.

15. The method of claim 13 , wherein calculating the probability further comprises:

independently estimating, for respective person, location, and keyword models, model parameters by iterative expectation and maximization operations; and

calculating the probability using the models in view of temporal continuity of the event and any overlap of two or more of the events.

16. The method of claim 13 , further comprising removing, from a document of the one or more representative documents, one or more of a duplicate image and an advertisement associated with any entity other than an entity associated with a web site from which the document was obtained.

17. The method of claim 13 , further comprising synchronizing the representative content of the storyboard layout with music.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2014
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 034543/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 5, 2006
From: LU, LIE; MA, WEI-YING; LI, ZHIWEI
To: MICROSOFT CORPORATION
Reel/Frame 017163/0941 →
Continuity (1)
Related Publication 20070094251A1 · Apr 26, 2007