Automatic discovery of business-specific terminology
A method and computer product encoding the method is available for preparing a domain or subdomain specific glossary. The method included using probabilities, word context, common terminology and different terminology to identify domain and subdomain specific language and a related glossary updated according to the method.
1. A computer product comprising computer executable code embodied in a non-transitory computer readable medium that, when executing on one or more computing devices performs a method of automatically extracting subdomain specific terminology and phrases within a language model for a language domain, comprising:
identifying terminology that is the same for all subdomains in the domain to define a common terminology;
subtracting the common terminology from the language model to identify subdomain-specific terminology for each subdomain;
calculating a priori appearance probability of a series of words using the subdomain terminology for each subdomain; and
automatically determining similarity of meanings of terms across subdomains by identifying terms with the similar appearance probability within a series of words.
2. The computer program product of claim 1 , wherein terms with a high appearance probability are deemed specific to a subdomain.
3. The computer product of claim 1 , the method further comprising determining that subdomain-specific terminology having the same appearance probability and appearing in the same context have equivalent meaning.
4. The computer program product of claim 1 , the method further comprising comparing the appearance probability for each series of words to determine terms important to each subdomain.
5. The computer program product of claim 1 , wherein the calculating a priori appearance probably is calculated using a word lattice.
6. The computer program product of claim 1 , wherein the calculating a priori appearance probability is calculated using word embedding.
7. The computer program product of claim 1 , wherein the subdomain is a specific business within an industry and the domain is the industry.
8. The computer program product of claim 1 , wherein the subdomain is business units common in an industry and the domain is the industry.
9. The computer program product of claim 1 , the method further comprising extracting subdomain-specific technology.
10. The computer program product of claim 1 , the method further comprising populating subdomain-specific glossaries with the subdomain-specific terminology.
11. A method of automatically extracting subdomain specific terminology and phrases within a language model for a language domain, comprising:
identifying terminology that is the same for all subdomains in the domain to define a common terminology;
subtracting the common terminology from the language model to identify subdomain specific terminology for each subdomain;
calculating a priori appearance probability of a series of words using the subdomain terminology for each subdomain; and
automatically determining similarity of meanings of terms across subdomains by identifying terms with the similar appearance probability within a series of words.
12. The method of claim 11 , wherein terms with a high appearance probability are deemed important to a subdomain.
13. The method of claim 11 , further comprising determining that subdomain-specific terminology having the same appearance probability and appearing in the same context have equivalent meaning.
14. The method of claim 11 , comparing the appearance probability for each series of words to determine terms important to each subdomain.
15. The method of claim 11 , wherein the calculating a priori appearance probably is calculated using a word lattice.
16. The method of claim 11 , wherein the calculating a priori appearance probability is calculated using word embedding.
17. The method of claim 11 , wherein the subdomain is a specific business within an industry and the domain is the industry.
18. The method of claim 11 , wherein the subdomain is business units common in an industry and the domain is the industry.
19. The method of claim 11 , further comprising extracting subdomain-specific technology.
20. The method of claim 11 , further comprising populating subdomain-specific glossaries with the subdomain-specific terminology.