Terms for query expansion using unstructured data
A processor determines a set of terms frequently occurring within unstructured data associated with criteria of a query. The processor analyzes the unstructured data to match a term of the set of terms, to criteria of the query. Matching terms are retained and unmatched terms are checked for semantic similarity to terms of the criteria to determine an inferred match of the term of the unstructured data to terms of the criteria of the query. In response to determining an inferred match, the inferred match term is added to the second set of terms, and the processor compares the second set of terms to the terms of the criteria of the query, and removes matching terms, resulting in a third set of terms added to the set of criteria terms of the query. The additional query terms will enhance the recall without diluting the precision.
1. A method for determining additional terms to expand recall, and maintaining precision of a query, the method comprising:
receiving a set of criteria terms of a query comprised of unstructured data;
accessing, by a processor, a source of unstructured training data that includes labelled training data comprised of terms;
generating, by the processor, a first set of terms from the unstructured training data, by frequency pattern mining of terms within the unstructured training data meeting or exceeding a pre-determined frequency of occurrence threshold;
extracting from the unstructured training data, one or more additional terms in response to a determination that the one or more additional terms fails to match a term of the first set of terms;
performing a semantic analysis on the one or more additional terms extracted from the unstructured training data;
generating, by the processor, a second set of terms by adding one or more additional terms extracted from the unstructured training data to the first set of terms, in response to a determination that the one or more additional terms of the unstructured training data are an inferred match and are semantically related to a term of the set of criteria terms of the query;
generating, by the processor, a third set of terms, based on removing each term of the second set of terms matching a term of the set of criteria terms of the query; and
generating, by the processor, an expanded recall, and maintaining precision of the query, based on adding the third set of terms to the set of criteria terms of the query.
2. The method of claim 1 , wherein a term of the first set of terms of unstructured training data is text based evidence data.
3. The method of claim 1 , wherein a selection of a term within the first set of terms of the unstructured training data for semantic analysis is based on determining that the term of the first set of terms fails to match a term of the second set of terms.
4. The method of claim 1 , wherein the semantic analysis includes determining a term most similar to a term of the set of criteria terms of the query, for an instance of unstructured training data in which multiple terms have an inferred match to the set of criteria terms of the query.
5. The method of claim 1 , wherein the semantic analysis is performed using a knowledge base of terms related to the set of criteria terms of the query.
6. The method of claim 1 , wherein the query is a criteria validation query requiring a decision of whether a condition of the criteria terms of the query is met.
7. The method of claim 1 , wherein the unstructured training data is manually generated evidence data for supervised training of criteria decision making functions.
8. The method of claim 1 , wherein a term of the unstructured training data is a phrase comprised of a combination of words and symbols.