Domain-specificity prediction for natural language processing

1 . A computer-implemented method for determining domain-specificity of a text term to improve precision of pre-trained language models in the computerized field of natural language processing, the method comprising:

receiving a plurality of domain-specific text corpora, wherein each of the plurality of domain-specific text corpora comprises a plurality of text documents associated with a respective domain;

training a set of subword-unit tokenizers with at least two different vocabulary sizes of the respective domain-specific text corpus;

receiving the text-term;

determining a domain-specificity fingerprint of the text-term, wherein the domain-specificity fingerprint comprises for each subword-unit tokenizer a number of subword-units required to represent the text-term; and

providing the domain-specificity fingerprint for determining the domain-specificity of the text term, wherein the domain-specificity of the text term is a measure of whether a term is used predominantly in one or more specific domains or whether it is used in a plurality of domains, wherein the domain-specificity of the text term is determined at least in part by analyzing a contour of the domain-specificity fingerprint and the contour comprises a variable corresponding to a respective subword-unit tokenizer of the domain at which a minimum number of subword units required to encode the text term across all domains is reached.

2 . The computer-implemented method of claim 1 , wherein the text term comprises words or multi-words.

3 . The computer-implemented method of claim 1 , wherein determining the domain-specificity fingerprint further comprises:

assigning the text term to one or more domains of the respective set of subword-unit tokenizers that compute a minimum number of subword-units required to represent the text-term with the smallest vocabulary size.

4 . The computer-implemented method of claim 1 , wherein determining the domain specificity by analyzing the contour further comprises:

computing the maximum delta between any pair of values in the contour; and

assigning the maximum delta as a score for the domain-specificity of the text term.

5 . The computer-implemented method of claim 2 , wherein the domain-specificity fingerprint of the multi-words are based on a domain provenance of the text-term using a k-nearest neighbor algorithm (k-NN) classifier.

6 . A computer program product for determining domain-specificity of a text term to improve precision of pre-trained language models in the computerized field of natural language processing, the computer program product comprising:

one or more non-transitory computer-readable storage media; and

program instructions stored on the one or more non-transitory computer-readable storage media, the program instructions to perform-operations comprising:

receiving a plurality of domain-specific text corpora, wherein each of the plurality of domain-specific text corpora comprises a plurality of text documents associated with a respective domain;

training a set of subword-unit tokenizers with at least two different vocabulary sizes of the respective domain-specific text corpus;

receiving the text-term;

determining a domain-specificity fingerprint of the text-term, wherein the domain-specificity fingerprint comprises for each subword-unit tokenizer a number of subword-units required to represent the text-term; and

providing the domain-specificity fingerprint for determining the domain-specificity of the text term, wherein the domain-specificity of the text term is a measure of whether a term is used predominantly in one or more specific domains or whether it is used in a plurality of domains and the domain specificity is determined by analyzing a contour of the domain-specificity fingerprint by computing a maximum delta between any pair of values in the contour and assigning the maximum delta as a score for the domain-specificity of the text term.

7 . The computer program product of claim 6 , wherein the text term comprises words or multi-words.

8 . The computer program product of claim 6 , wherein determining the domain-specificity fingerprint further comprises:

assigning the text term to one or more domains of the respective set of subword-unit tokenizers that compute a minimum number of subword-units required to represent the text-term with the smallest vocabulary size.

9 . The computer program product of claim 6 , wherein the contour comprises a value corresponding to a respective subword-unit tokenizer of the domain at which a minimum number of subword units required to encode the text term across all domains and vocabulary sizes is reached.

10 . The computer program product of claim 7 , wherein the domain-specificity fingerprint of the multi-words are based on a domain provenance of the text-term using a k-nearest neighbor algorithm (k-NN) classifier.

11 . A computer system for determining domain-specificity of a text term to improve precision of pre-trained language models in the computerized field of natural language processing, the computer system comprising:

one or more computer processors;

one or more computer readable storage media; and

program instructions stored on the computer readable storage media for execution by at least one of the one or more processors, the program instructions comprising:

program instructions to receive a plurality of domain-specific text corpora, wherein each of the plurality of domain-specific text corpora comprises a plurality of text documents associated with a respective domain;

program instructions to train a set of subword-unit tokenizers with at least two different vocabulary sizes of the respective domain-specific text corpus;

program instructions to receive the text-term;

program instructions to determine a domain-specificity fingerprint of the text-term, wherein the domain-specificity fingerprint comprises for each subword-unit tokenizer a number of subword-units required to represent the text-term; and

program instructions to provide the domain-specificity fingerprint for determining the domain-specificity of the text term, wherein the domain-specificity of the text term is a measure of whether a term is used predominantly in one or more specific domains or whether it is used in a plurality of domains, wherein the domain-specificity of the text term is determined at least in part by analyzing a contour of the domain-specificity fingerprint and the contour comprises a variable corresponding to a respective subword-unit tokenizer of the domain.

12 . The computer system of claim 11 , wherein the text term comprises words or multi-words.

13 . The computer system of claim 11 , wherein determining the domain-specificity fingerprint further comprises:

program instructions to assign the text term to one or more domains of the respective set of subword-unit tokenizers that compute a minimum number of subword-units required to represent the text-term with the smallest vocabulary size.

14 . The computer system of claim 11 , wherein determining the domain specificity by analyzing the contour further comprises:

program instructions to compute the maximum delta between any pair of values in the contour; and

program instructions to assign the maximum delta as a score for the domain-specificity of the text term.