Method and system for searching phrase concepts in documents
A system and method for fast concept search in multiple documents where the concept is expressed by plurality of words, all of which have to be in the same sentence and within specified range. The system automatically finds equivalent expressions of the same concept, and returns as search results all documents in which the concept is contained.
1. A method for performing search to retrieve phrase concepts from documents stored in a corpus, the method is comprised of the following steps:
a. Splitting all documents in the search domain into sentences; splitting each sentence to its words; keeping for each word its phonetic representation and its indexes;
b. receiving a query regarding the search subject from the user, wherein the logic of the query allows the user to specify that certain search phrases must be found within the boundaries of one sentence;
c. finding syntactic and semantic synonyms to all words of the query;
d. preparing set of logical queries for all synonym combinations;
e. retrieving all sentences that respond to at least one query;
f. calculating a score for each retrieved sentence, and
g. displaying documents that contain sentences having a score higher than a predefined threshold.
2. The method according to claim 1 wherein semantic synonyms are derived from an ontology database.
3. The method according to claim 1 wherein the syntactic synonyms are derived from the words stored in the corpus, by finding similar phonetic representation between a word in the query and a word in the corpus and measuring the distance between these two words.
4. The method according to claim 3 wherein Jaro-Winkler algorithm is used to compute the distance between words having similar phonetic representation.
5. The method according to claim 1 wherein the user can update the ontology.
6. The method according to claim 3 wherein the Jaro-Winkler algorithm is adapted to the Hebrew Language.
7. The method according to claim 1 wherein sentence splitting is based on syntactic analysis and noun-phrase analysis.
8. A system comprising one or more computers configured to perform operations for retrieving findings from documents stored in a corpus, operations comprising:
a. Splitting all documents in the search domain into sentences; splitting each sentence to its words; keeping for each word its phonetic representation and its indexes;
b. receiving a query regarding the search subject from the user, wherein the logic of the query allows the user to specify that certain search phrases must be found within the boundaries of one sentence;
c. finding syntactic and semantic synonyms to all words of the query;
d. preparing set of logical queries for all synonym combinations;
e. retrieving all sentences that respond to at least one query;
f. calculating a score for each retrieved sentence, and
g. displaying documents that contain sentences having a score higher than a predefined threshold.