Method for cross-domain feature correlation
A method for correlating information across distinct domains without requiring feature co-occurrence. The disparate information collections are broken down into features, and a correlation index with correlation score is created. To determine the correlation between distinct domains, an information artifact collection is reduced to a representational set of features, these features are replaced with correlated features using the correlation index, and the new set of features is matched against the second information artifact collection using an appropriate comparison technique. The correlation method allows a single input artifact to be matched against an existing collection, resulting in a set of correlated artifacts from the disparate collection, each ranked by correlation score.
1. A method for data correlation comprising the steps of:
providing a first correlation between a first information artifact collection and a second information artifact collection;
extracting at least a first feature from a second information artifact in said first information artifact collection and at least a first feature from a second information artifact in said second information artifact collection, wherein said extraction step comprises the steps of: (i) identifying at least a first lexical category in a second information artifact in said first information artifact collection; (ii) identifying at least a first phrase in said information artifact; (iii) normalizing at least a first term in said information artifact; (iv) removing all features except for nouns, verbs, and phrases from said information artifact; and (v) removing each feature that occurs only once in said information artifact;
selecting at least said first extracted features from said first and second information artifact collections;
creating a correlation index between said selected features, wherein said creating step comprises the steps of: (i) pairing at least a first feature extracted from said second information artifact in said first information artifact collection with at least a first feature extracted from a second information artifact in said second information artifact collection; and (ii) calculating a correlation score between said paired features;
extracting at least a first feature from a third information artifact in said first information artifact collection;
applying said correlation index to said first feature extracted from said third information artifact; and
computing a correlation score between said first feature extracted from said third information artifact and at least a first feature from one or more information artifacts in said second information artifact collection.
2. The method according to claim 1 , wherein said extracting steps use said first correlation to identify which features to extract.
3. The method according to claim 1 , wherein said first correlation links a feature from a first information artifact in said first information artifact collection to at least a first feature from a first information artifact in said second information artifact collection.
4. The method according to claim 1 , wherein said extracting steps comprise at least a first set of extraction rules.
5. The method according to claim 4 , wherein said extracting steps comprise a set of extraction rules for each of said first and second information artifact collections.
6. The method according to claim 1 , wherein said selecting step comprises selecting every feature extracted from said first and second information artifact collections.
7. The method according to claim 1 , wherein said selecting step further comprises the step of:
filtering said extracted features using a first filter.
8. The method according to claim 7 , wherein said first filter is a term frequency—inverse document frequency filtering method.
9. The method according to claim 1 , wherein said correlation index comprises a correlation table, said correlation table further comprising said paired features and said calculated correlation score.
10. The method according to claim 1 , wherein said applying step further comprises the steps of:
comparing said extracted feature from said third information artifact to each feature in said correlation index; and
matching said extracted feature from said third information artifact to one feature from at least one paired feature in said correlation index.
11. The method according to claim 10 , wherein said applying step further comprises the step of:
retrieving the matching feature pair from said correlation index.
12. The method according to claim 11 , wherein said method for data correlation further comprises the steps of:
using said computed correlation score to rank each retrieved feature.
13. A non-transitory computer-readable storage medium containing program code comprising:
program code for providing a first correlation between a first information artifact collection and a second information artifact collection;
program code for extracting at least a first feature from a second information artifact in said first information artifact collection and at least a first feature from a second information artifact in said second information artifact collection;
program code for selecting at least said first extracted features from said first and second information artifact collections;
program code for creating a correlation index between said selected features, wherein said program code for creating a correlation index further comprises the steps of: (i) pairing at least a first feature extracted from said second information artifact in said first information artifact collection with at least a first feature extracted from a second information artifact in said second information artifact collection; and (ii) calculating a correlation score between said paired features;
program code for extracting at least a first feature from a third information artifact in said first information artifact collection;
program code for applying said correlation index to said first feature extracted from said third information artifact, wherein said correlation index comprises a correlation table, said correlation table further comprising said paired features and said calculated correlation score, and wherein said program code for applying said correlation index to said first feature extracted from said third information artifact further comprises the steps of: (i) comparing said extracted feature from said third information artifact to each feature in said correlation index; and matching said extracted feature from said third information artifact to one feature from at least one paired feature in said correlation index; and
program code for computing a correlation score between said first feature extracted from said third information artifact and at least a first feature from one or more information artifacts in said second information artifact collection.