System and method for summarizing a multimedia content item
A multimedia content item is summarized based on its audio track and a desired compression budget. The audio track is extracted and processed by an automatic speech recognizer to obtain a time-aligned text transcript. The text-transcript is partitioned into a plurality of segment sequences. An informativeness score based on a salience score and a diversity score is computed for each of the segments. A coherence score is also computed for the segments in the plurality of sequences. A subsequence of one of the segment sequences that optimizes for informativeness and coherence is selected for generating a new content item summarizing the multimedia content item.
1. A method comprising:
identifying, by a computing device, a multimedia content item comprising digital content and an audio track;
analyzing, via the computing device, the multimedia content item, and based on said analysis, identifying a compression budget, said compression budget comprising information indicating a runtime value of the multimedia content item and a maximum percentage of a total of the runtime value of the multimedia content item;
determining, via the computing device, that the maximum percentage within the compression budget is at least a threshold fraction less than the runtime value;
partitioning, via the computing device, upon said determination, said multimedia content item based on the runtime value and the maximum percentage of the total of the runtime value indicated in said compression budget;
identifying, via the computing device, based on said partitioning, segments of digital content and audio information, the segments corresponding to a portion of the multimedia content item;
generating, via the computing device, a summary content item based on said partitioning, said summary content item comprising said identified segments of the digital content and audio information;
obtaining a time-aligned text transcript of the audio track;
constructing a graph representing the segments, wherein each segment is a node in the graph;
computing an informativeness score for each segment based on a corresponding salience score and diversity score; and
selecting segments for the summary content item based on the informativeness scores.
2. The method of claim 1 , wherein said compression budget indicates a value for a runtime of the summary content item to be generated, said runtime of the summary content item to be generated being a target length of the summary content item.
3. The method of claim 2 , further comprising: identifying a subset of said segments of digital content and audio information, said subset identification based on the runtime of the summary content item indicated by the compression budget.
4. The method of claim 1 , wherein said analysis further comprises: analyzing said multimedia content item by applying a linear regression algorithm, and determining, based on said application, whether said multimedia content item can be summarized, wherein said generation is based on said determination.
5. The method of claim 4 , wherein said determination is further based on a compression rate of said multimedia content item.
6. The method of claim 1 , wherein said analysis further comprises: identifying a length of the digital content, a source of the digital content and information indicating coarse-grained features of the digital content and the audio track.
7. The method of claim 1 , wherein said each segment of the digital content includes information indicating time intervals within the multimedia content item, wherein the time intervals are a sequence of non-contiguous, non-overlapping sub-intervals of equal lengths.
8. A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions, that when executed by a computing device, perform a method comprising:
identifying, by the computing device, a multimedia content item comprising digital content and an audio track;
analyzing, via the computing device, the multimedia content item, and based on said analysis, identifying a compression budget, said compression budget comprising information indicating a runtime value of the multimedia content item and a maximum percentage of a total of the runtime value of the multimedia content item;
determining, via the computing device, that the maximum percentage within the compression budget is at least a threshold fraction less than the runtime value;
partitioning, via the computing device, upon said determination, said multimedia content item based on the runtime value and the maximum percentage of the total of the runtime value indicated in said compression budget;
identifying, via the computing device, based on said partitioning, segments of digital content and audio information, the segments corresponding to a portion of the multimedia content item;
generating, via the computing device, a summary content item based on said partitioning, said summary content item comprising said identified segments of the digital content and audio information;
obtaining a time-aligned text transcript of the audio track;
constructing a graph representing the segments, wherein each segment is a node in the graph;
computing an informativeness score for each segment based on a corresponding salience score and diversity score; and
selecting segments for the summary content item based on the informativeness scores.
9. The non-transitory computer-readable storage medium of claim 8 , wherein said compression budget indicates a value for a runtime of the summary content item to be generated, said runtime of the summary content item to be generated being a target length of the summary content item.
10. The non-transitory computer-readable storage medium of claim 9 , further comprising: identifying a subset of said segments of digital content and audio information, said subset identification based on the runtime of the summary content item indicated by the compression budget.
11. The non-transitory computer-readable storage medium of claim 8 , wherein said analysis further comprises: analyzing said multimedia content item by applying a linear regression algorithm, and determining, based on said application, whether said multimedia content item can be summarized, wherein said generation is based on said determination.
12. The non-transitory computer-readable storage medium of claim 11 , wherein said determination is further based on a compression rate of said multimedia content item.
13. The non-transitory computer-readable storage medium of claim 8 , wherein said analysis further comprises: identifying a length of the digital content, a source of the digital content and information indicating coarse-grained features of the digital content and the audio track.
14. The non-transitory computer-readable storage medium of claim 8 , wherein said each segment of the digital content includes information indicating time intervals within the multimedia content item, wherein the time intervals are a sequence of non-contiguous, non-overlapping sub-intervals of equal lengths.
15. A computing device comprising:
a processor; and
a non-transitory computer-readable storage medium for tangibly storing thereon program logic for execution by the processor, the program logic comprising:
logic executed by the processor for identifying a multimedia content item comprising digital content and an audio track;
logic executed by the processor for analyzing the multimedia content item, and based on said analysis, identifying a compression budget, said compression budget comprising information indicating a runtime value of the multimedia content item and a maximum percentage of a total of the runtime value of the multimedia content item;
logic executed by the processor for determining that the maximum percentage within the compression budget is at least a threshold fraction less than the runtime value;
logic executed by the processor for partitioning, upon said determination, said multimedia content item based on the runtime value and the maximum percentage of the total of the runtime value indicated in said compression budget;
logic executed by the processor for identifying, based on said partitioning, segments of digital content and audio information, the segments corresponding to a portion of the multimedia content item;
logic executed by the processor for generating a summary content item based on said partitioning, said summary content item comprising said identified segments of the digital content and audio information;
logic executed by the processor for obtaining a time-aligned text transcript of the audio track;
logic executed by the processor for constructing a graph representing the segments, wherein each segment is a node in the graph;
logic executed by the processor for computing an informativeness score for each segment based on a corresponding salience score and diversity score; and
logic executed by the processor for selecting segments for the summary content item based on the informativeness scores.