Category-sensitive ranking for text
View Patent ↗Provided are methods, systems and apparatus which include computer program products, for generating topic models for text summarization In one aspect, a method includes receiving a first document of text that is associated with one or more category labels and that includes one or more sequences of one or more words, determining a category label that represents a first category associated with the first document, sampling the one or more sequences to determine a topic and a co-occurrence relationship between the topic and the category label, where a topic represents a subdivision within a category, sampling the one or more sequences to determine a co-occurrence relationship between a sequence in the first document and the topic, and generating a category-topic model that represents the co-occurrence relationships.
1. A method comprising:
receiving a plurality of documents of text, wherein each document is associated with one or more category labels and includes one or more sequences of one or more words;
determining a plurality of topics from the plurality of documents, wherein each topic represents a subdivision of a respective category label;
performing a plurality of sampling iterations to generate a category-topic model that represents co-occurrence relationships between sequences and topics and co-occurrence relationships between topics and categories, wherein performing each of the plurality of sampling iterations comprises, for each sequence in each of the plurality of documents:
sampling a category label for the sequence from the category labels associated with the document that includes the sequence;
sampling a topic for the sequence; and
updating current values of representations of the co-occurrence relationships based on the category label and the topic sampled for the sequence.
2. The method of claim 1 , where generating a category-topic model further includes:
determining, from the co-occurrence relationship between the topic and the category label, a first probability of the topic being associated with a document given that the category label is associated with the document; and
determining, from a co-occurrence relationship between the sequence and the topic, a second probability of the sequence occurring in the document given that the topic is associated with the document.
3. The method of claim 2 , where sampling the category label and the topic comprises sampling the category label and the topic from a distribution that satisfies:
P
(
u
d
,
i
=
c
,
z
d
,
i
=
k
❘
w
d
,
i
=
v
,
w
d
,
-
i
,
z
d
,
-
i
,
u
d
,
-
i
,
y
d
,
α
,
β
)
=
C
v
,
k
WT
+
β
∑
v
′
=
1
V
C
v
,
k
WT
+
V
β
C
k
,
c
TC
+
α
∑
k
′
=
1
K
C
k
,
c
TC
+
K
α
;
where u d,i is a category label; c is a particular category; z d,i is a topic; k is a particular topic; w d,i is a sequence of one or more words; v is a particular sequence of one or more words;
C v,k WT is a number of times that topic k is assigned to sequence v , not including a current instance of sampling i; C k,c TC is a number of times that topic k is assigned to category c, not including the current instance of sampling i; α is a constant; β is a constant; and d is a document.
4. The method of claim 3 , wherein performing the plurality of sampling iterations comprises performing a respective portion of the sampling independently on each of multiple processors, and wherein each processor p calculates an update ΔC |p WT and an update ΔC |p TC , and C k,c TC and C v,k WT are updated according to the expressions
C
WT
+=
∑
p
=
1
P
Δ
C
❘
p
WT
and
C
TC
+=
∑
p
=
1
P
Δ
C
❘
p
TC
.
5. The method of claim 3 , where the determining the first and second probabilities includes:
calculating the conditional probabilities that the topic is k given that the category label is c, the sequence is v given that the topic is k, the category label is c given that the topic is k, and the topic is k given that the sequence is v.
6. The method of claim 5 , where the conditional probabilities satisfy the expressions:
P
(
z
=
k
❘
u
=
c
)
=
C
k
,
c
TC
+
α
∑
k
′
C
k
′
,
c
TC
+
K
α
;
P
(
w
=
v
❘
z
=
k
)
=
C
v
,
k
WT
+
β
∑
v
′
C
v
′
,
k
WT
+
V
β
;
P
(
u
=
c
❘
z
=
k
)
=
C
k
,
c
TC
+
α
∑
c
′
C
k
,
c
′
TC
+
C
α
;
and
P
(
z
=
k
❘
w
=
v
)
=
C
v
,
k
WT
+
β
∑
k
′
C
v
,
k
′
WT
+
K
β
.
7. The method of claim 1 , further comprising:
receiving a textual input;
determining in the textual input one or more sequences of one or more words that are each associated with a respective category;
ranking each of the sequences using the category-topic model;
identifying one or more of the determined sequences that have a ranking greater than a threshold ranking value; and
generating a summary of the textual input that includes the identified sequences.
8. The method of claim 7 , where the ranking includes calculating a probabilistic characteristic value for each sequence of one or more words, where the probabilistic characteristic value is expressed as:
P
(
w
❘
c
)
P
(
c
❘
w
)
=
[
∑
z
=
1
K
P
(
w
❘
z
)
P
(
z
❘
c
)
]
[
∑
z
=
1
K
P
(
z
❘
w
)
P
(
c
❘
z
)
]
,
where w is a sequence of one or more words, c is a category, and z is a topic.
9. The method of claim 1 , where the category labels are human-readable.
10. The method of claim 1 , where at least one of the topics is not human-readable.
11. The method of claim 1 , where a sequence of one or more words is a phrase, sentence, or paragraph.
12. The method of claim 1 , further comprising:
determining that a first document of the plurality of documents is not associated with any category labels; and
assigning a unique label to the first document.
13. A system comprising:
one or more computer-readable storage devices including computer program instructions; and
one or more computers operable to execute to execute the instructions to perform operations comprising:
receiving a plurality of documents of text, wherein each document is associated with one or more category labels and includes one or more sequences of one or more words;
determining a plurality of topics from the plurality of documents, wherein each topic represents a subdivision of a respective category label;
performing a plurality of sampling iterations to generate a category-topic model that represents co-occurrence relationships between sequences and topics and co-occurrence relationships between topics and categories, wherein performing each of the plurality of sampling iterations comprises, for each sequence in each of the plurality of documents:
sampling a category label for the sequence from the category labels associated with the document that includes the sequence;
sampling a topic for the sequence; and
updating current values of representations of the co-occurrence relationships based on the category label and the topic sampled for the sequence.
14. The system of claim 13 , where sampling the category label and the topic comprises sampling the category label and the topic from a distribution that satisfies:
P
(
u
d
,
i
=
c
,
z
d
,
i
=
k
|
w
d
,
i
=
v
,
w
d
,
-
i
,
z
d
,
-
i
,
u
d
,
-
i
,
y
d
,
α
,
β
)
=
C
v
,
k
W
T
+
β
∑
v
′
=
1
V
C
v
,
k
WT
+
V
β
C
k
,
c
TC
+
α
∑
k
′
=
1
K
C
k
,
c
TC
+
K
α
;
where u d,i is a category label; c is a particular category;z d,i is a topic; k is a particular topic; w d,i is a sequence of one or more words; v is a particular sequence of one or more words;
C v,k WT is a number of times that topic k is assigned to sequence v , not including a current instance of sampling i; C k,c TC is a number of times that topic k is assigned to category c, not including the current instance of sampling i; α is a constant; β is a constant; and d is a document.
15. The system of claim 14 , wherein performing the plurality of sampling iterations comprises performing a respective portion of the sampling independently on each of multiple processors, and wherein each processor p calculates an update ΔC |p WT and an update ΔC |p TC , and C TC k,c and C v,k WT are updated according to the expressions
C
WT
+=
∑
p
=
1
P
Δ
C
|
p
WT
and
C
TC
+=
∑
p
=
1
P
Δ
C
|
p
TC
.
16. The system of claim 15 , where conditional probabilities satisfy the expressions:
P
(
z
=
k
|
u
=
c
)
=
C
k
,
c
TC
+
α
∑
k
′
C
k
′
,
c
TC
+
K
α
;
P
(
w
=
v
|
z
=
k
)
=
C
v
,
k
W
T
+
β
∑
v
′
C
v
′
,
k
WT
+
V
β
;
P
(
u
=
c
|
z
=
k
)
=
C
k
,
c
TC
+
α
∑
c
′
C
k
,
c
′
TC
+
C
α
;
and
P
(
z
=
k
|
w
=
v
)
=
C
v
,
k
WT
+
β
∑
k
′
C
v
,
k
WT
+
K
β
.
17. One or more non-transitory computer-readable storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:
receiving a plurality of documents of text, wherein each document is associated with one or more category labels and includes one or more sequences of one or more words;
determining a plurality of topics from the plurality of documents, wherein each topic represents a subdivision of a respective category label;
performing a plurality of sampling iterations to generate a category-topic model that represents co-occurrence relationships between sequences and topics and co-occurrence relationships between topics and categories, wherein performing each of the plurality of sampling iterations comprises, for each sequence in each of the plurality of documents:
sampling a category label for the sequence from the category labels associated with the document that includes the sequence;
sampling a topic for the sequence; and
updating current values of representations of the co-occurrence relationships based on the category label and the topic sampled for the sequence.
18. The computer-readable storage media of claim 17 , where sampling the category label and the topic comprises sampling the category label and the topic from a distribution that satisfies:
P
(
u
d
,
i
=
c
,
z
d
,
i
=
k
|
w
d
,
i
=
v
,
w
d
,
-
i
,
z
d
,
-
i
,
u
d
,
-
i
,
y
d
,
α
,
β
)
=
C
v
,
k
W
T
+
β
∑
v
′
=
1
V
C
v
,
k
WT
+
V
β
C
k
,
c
TC
+
α
∑
k
′
=
1
K
C
k
,
c
TC
+
K
α
;
where u d,i is a category label; c is a particular category;z d,i is a topic; k is a particular topic; w d,i is a sequence of one or more words; v is a particular sequence of one or more words; C v,k WT is a number of times that topic k is assigned to sequence v , not including a current instance of sampling i; C k,c TC is a number of times that topic k is assigned to category c, not including the current instance of sampling i; α is a constant; β is a constant; and d is a document.
19. The computer-readable storage media of claim 18 , wherein performing the plurality of sampling iterations comprises performing a respective portion of the sampling independently on each of multiple processors, and wherein each processor p calculates an update ΔC |p WT and an update ΔC |p TC , and C k,c TC and C v,k WT are updated according to the expressions
C
WT
+=
∑
p
=
1
P
Δ
C
|
p
WT
and
C
TC
+=
∑
p
=
1
P
Δ
C
|
p
TC
.
20. The computer-readable storage media of claim 19 , where conditional probabilities satisfy the expressions:
P
(
z
=
k
|
u
=
c
)
=
C
k
,
c
TC
+
α
∑
k
′
C
k
′
,
c
TC
+
K
α
;
P
(
w
=
v
|
z
=
k
)
=
C
v
,
k
W
T
+
β
∑
v
′
C
v
′
,
k
WT
+
V
β
;
P
(
u
=
c
|
z
=
k
)
=
C
k
,
c
TC
+
α
∑
c
′
C
k
,
c
′
TC
+
C
α
;
and
P
(
z
=
k
|
w
=
v
)
=
C
v
,
k
WT
+
β
∑
k
′
C
v
,
k
WT
+
K
β
.