Automatic extraction of semantically similar question topics
A method, computer system, and computer program product are provided for automatically extracting semantically-similar question topics. A set of documents, wherein each document includes a plurality of words. One or more clusters of documents are identified in the set of documents based on a presence of common words in the documents of the one or more clusters. The one or more clusters are adjusted based on semantic similarity by adding or removing one or more documents from the one or more clusters. A topic is extracted from each adjusted cluster of documents.
1 . A computer-implemented method comprising:
obtaining, via at least one processor, a set of documents submitted by participants from a communication session conducted over a network, wherein each document is in electronic form and includes a plurality of words;
identifying, via the at least one processor, one or more clusters of documents in the set of documents based on a presence of a threshold number of common words in documents of the one or more clusters of documents;
adjusting the one or more clusters of documents, via the at least one processor, based on semantic similarity by adding or removing one or more documents from a cluster based on the one or more documents being within a threshold distance from the cluster, wherein the threshold distance is based on a plurality of adjustable values;
extracting, via the at least one processor, a topic from each adjusted cluster of documents;
determining, via the at least one processor, a document from each adjusted cluster of documents containing the topic;
sending, via the at least one processor, the document from each adjusted cluster of documents to a participant of the communication session to produce a response pertaining to the topic from each adjusted cluster of documents, wherein sending the document from each adjusted cluster enables the response to be produced for remaining documents in each adjusted cluster rather than sending individual documents of each adjusted cluster;
presenting, via the at least one processor, the response to the participants in the communication session, and
adjusting, via the at least one processor, the threshold distance for adjustment of document clusters by modifying the plurality of adjustable values based on user feedback for the topic from each adjusted cluster of documents.
2 . The computer-implemented method of claim 1 , wherein each document is a question submitted by a participant of a question-and-answer session.
3 . The computer-implemented method of claim 1 , wherein extracting the topic from a particular cluster includes identifying a particular document having a longest sequence of stemmed words in common with a majority of other documents of the particular cluster.
4 . The computer-implemented method of claim 1 , wherein adding or removing the one or more documents from a cluster comprises generating a representation of the set of documents in a vector space, identifying a centroid for the cluster in the vector space, and adding or removing the one or more documents based on the one or more documents being within the threshold distance from the centroid.
5 . The computer-implemented method of claim 1 , further comprising:
obtaining the user feedback in response to presenting the topic to one or more users.
6 . The computer-implemented method of claim 1 , further comprising presenting the topic to one or more users via a display.
7 . The computer-implemented method of claim 1 , wherein the plurality of words in each document are tokenized prior to identifying the one or more clusters of documents.
8 . A computer system comprising:
one or more computer processors;
one or more computer readable storage media; and
program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, the program instructions comprising instructions to perform operations including:
obtaining a set of documents submitted by participants from communication session conducted over a network, wherein each document is in electronic form and includes a plurality of words;
identifying one or more clusters of documents in the set of documents based on a presence of a threshold number of common words in documents of the one or more clusters of documents;
adjusting the one or more clusters of documents based on semantic similarity by adding or removing one or more documents from a cluster based on the one or more documents being within a threshold distance from the cluster, wherein the threshold distance is based on a plurality of adjustable values;
extracting a topic from each adjusted cluster of documents;
determining a document from each adjusted cluster of documents containing the topic:
sending the document from each adjusted cluster of documents to a participant of the communication session to produce a response pertaining to the topic from each adjusted cluster of documents, wherein sending the document from each adjusted cluster enables the response to be produced for remaining documents in each adjusted cluster rather than sending individual documents of each adjusted cluster:
presenting the response to the participants in the communication session, and
adjusting the threshold distance for adjustment of document clusters by modifying the plurality of adjustable values based on user feedback for the topic from each adjusted cluster of documents.
9 . The computer system of claim 8 , wherein each document is a question submitted by a participant of a question-and-answer session.
10 . The computer system of claim 8 , wherein the am instructions to extract the topic from a particular cluster include program instructions to perform further operations including a particular document having a longest sequence of stemmed words in common with a majority of other documents of the particular cluster.
11 . The computer system of claim 8 , wherein adding or removing the one or more documents from a cluster comprises generating a representation of the set of documents in a vector space, identifying a centroid for the cluster, and adding or removing the one or more documents based on the one or more documents being within the threshold distance from the centroid.
12 . The computer system of claim 8 , wherein the program instructions further comprise program instructions to perform further operations including:
obtaining the user feedback in response to presenting the topic to one or more users.
13 . The computer system of claim 8 , further comprising program instructions to perform further operations including presenting the topic to one or more users via a display.
14 . The computer system of claim 8 , wherein the plurality of words in each document are tokenized prior to identifying the one or more clusters of documents.
15 . One or more non-transitory computer readable storage media encoded with program instructions that when executed by a computer, cause the computer to perform operations including:
obtaining a set of documents submitted by participants from a communication session conducted over a network, wherein each document is in electronic form and includes a plurality of words;
identifying one or more clusters of documents in the set of documents based on a presence of a threshold number of common words in documents of the one or more clusters of documents;
adjusting the one or more clusters of documents based on semantic similarity by adding or removing one or more documents from a cluster based on the one or more documents being within a threshold distance from the cluster, wherein the threshold distance is based on a plurality of adjustable values;
extracting a topic from each adjusted cluster of documents;
determining document from each adjusted cluster of documents containing the topic;
sending the document from each adjusted cluster of documents to a participant of the communication session to produce a response pertaining to the topic from each adjusted cluster of documents, wherein sending the document from each adjusted cluster enables the response to be produced for remaining documents in each adjusted cluster rather than sending individual documents of each adjusted cluster:
presenting the response to the participants in the communication session; and
adjusting the threshold distance for adjustment of document clusters by modifying the plurality of adjustable values based on user feedback for the topic from each adjusted cluster of documents.
16 . The one or more non-transitory computer readable storage media of claim 15 , wherein each document is a question submitted by a participant of a question-and-answer session.
17 . The computer one or more non-transitory computer readable storage media of claim 15 , wherein the program instructions to extract the topic from a particular cluster further cause the computer to perform further operations including identifying a particular document having a longest sequence of stemmed words in common with a majority of other documents of the particular cluster.
18 . The one or more non-transitory computer readable storage media of claim 15 , wherein adding or removing the one or more documents from a cluster comprises generating a representation of the set of documents in a vector space, identifying a centroid for the cluster, and adding or removing the one or more documents based on the one or more documents being within the threshold distance from the centroid.
19 . The one or more non-transitory computer readable storage media of claim 5 , wherein the program instructions further cause the computer to perform further operations including:
obtaining the user feedback in response to presenting the topic to one or more users.
20 . The one or more non-transitory computer readable storage media of claim 15 , wherein the program instructions further cause the computer to perform further operations including presenting the topic to one or more users via a display.