Method and apparatus for highlighting diverse aspects in a document
The disclosure generally relates to document summarization. Given a document, summarization can be defined as picking k sentences from the original document D such that the constructed summary exhibits two key properties: coverage and orthogonality. In one embodiment of the disclosure, the two requirements are captured in a combinatorial formulation of the problem and presented as an algorithms.
1. A computer-implemented method for creating a summary of one or more electronic documents, the summary consisting of sentences that span a portion of a spectrum of aspects discussed in the one or more electronic documents and capturing a different aspect of the document, the method comprising:
at a processor circuit in communication with a database, combining information extracted from said one or more electronic documents into a single plain text electronic document;
filtering the plain text electronic document to remove stop words and spam words and linking semantically similar words and phrases, to construct a filtered electronic document D having n sentences and m unique words;
identifying a subset of the n sentences for a predetermined summary length, the subset summarizing filtered electronic document D and identifying different aspects of the document D,
wherein the summary length is an integer and wherein the size of the subset is less than or equal to the predetermined summary length;
optimizing a combinatorial function defined C f in which W x , is a set of unique words/phrases, where every word in this set W x , appears in exactly x sentences in S; and D is a matrix, k is an integer, S is a subset of the columns of D with |S|<=k such that the following function is maximized
C
f
(
D
[
S
]
)
=
∑
x
=
0
S
∑
w
∈
W
x
f
(
x
)
,
wherein ƒ is a function defined as
f
(
x
)
=
{
0
if
x
=
0
1
2
x
-
1
if
x
>
0
or
f
(
x
)
=
{
0
if
x
=
0
1
if
x
>
0
;
and wherein 0≦χ≦|S|.