Latent semantic analysis for application in a question answer system
View Patent ↗A system and method that improves obtaining similarity measure between concepts based on Latent Semantic Analysis by taking onto account graph structure derived from the knowledge bases by using a vector propagation algorithm, in the context domain, such as a medical domain. Concepts contained in a corpus of documents are expressed in a graph wherein each node is a concept and edges between node express relation between concepts weighted by the number of semantic relations determined from the corpus. A vector of neighbors is created and assigned to each concept, thereby providing an improved similarity measure between documents, i.e., corpus and query against corpus.
1. A method for estimating similarity between concepts, the method comprising:
receiving a set of concepts related to a corpus of text documents;
creating a representative graph structure having graph nodes each representing a latent semantic analysis (LSA) vector associated with a concept, and a node having one or more graph edges, each graph edge representing a strength of a relation between concepts based on an ontology; and
deriving, for a concept, a new or modified vector represented by a node in the graph by propagating the LSA vectors against said graph structure, said new or modified vector representing a modified estimated similarity between concepts,
wherein a programmed processor device is configured to perform said receiving, creating and deriving.
2. The method as claimed in claim 1 , wherein said propagating a vector of a concept against said graph structure includes averaging the LSA vectors neighboring said concept.
3. The method as claimed in claim 1 , wherein said corpus of text documents includes a knowledgebase (KB), said representative graph structure being generated from ontologies found in the KB.
4. The method as claimed in claim 3 , wherein said knowledgebase includes a medical knowledgebase.
5. The method as claimed in claim 3 , wherein said created graph structure is an undirected graph generated from ontologies found in the KB, said created graph showing the relations of concepts weighted by a number of relations, wherein each edge between related concepts is a weight representing the number of relations.
6. The method as claimed in claim 5 , wherein a semantic relation includes an ISA relation.
7. The method as claimed in claim 5 , wherein said undirected graph is derived by collapsing each of different relations between concepts in the knowledge base.
8. The method as claimed in claim 5 , wherein said vector propagating comprises:
for a known LSA concept vector c i , applying a formula against data of said undirected graph, said formula comprising:
c
->
i
′
=
c
->
i
+
∑
c
i
∈
N
(
c
)
c
->
i
w
(
c
,
c
i
)
d
(
c
i
)
_
_
where c is a neighbor vector concept, {right arrow over (c)} i , is the concept vector propagated against the graph, {right arrow over (c)}′ i is a new concept vector, w(c, c i ) is the number of semantic relations involving concepts c and c i , and d(c) is an average vector.
9. The method as claimed in claim 8 , wherein the applying said vector propagation formula comprises: given a concept vector “c”,
identifying the vectors for the concept in graph;
computing an average of those identified vectors, and
replacing the original concept vector c with new vector average.