Granted Patent B2

US 12,293,153 · App. 17/892,169 · Granted May 6, 2025

Inventors: Shlomit Ifergan Shachor (Yokneam Eilit, IL); Natalia Razinkov (Haifa, IL); Micha Gideon Moffie (Zichron Yaakov, IL); Omer Yehuda Boehm (Haifa, IL)

Assignee: International Business Machines Corporation

G06F40/284

Abstract

Claims (57)

1. A computer-implemented method comprising:

obtaining multiple glossary terms each comprising one or more words; automatically operating a fuzzy token generator to generate multiple fuzzy tokens from each word of each of the glossary terms;

automatically calculating a similarity score for each of the fuzzy tokens, wherein the similarity score denotes a similarity between the respective fuzzy token and its respective word;

obtaining multiple input terms to be matched with the multiple glossary terms;

automatically operating a tokenizer to separate each of the input terms into multiple input tokens;

automatically generating multiple n-grams from each of the input tokens;

automatically comparing the n-grams with the fuzzy tokens, to output a list of matching n-grams and fuzzy tokens;

based on the list of matching n-grams and fuzzy tokens, automatically identifying, from the glossary terms, candidate glossary term matches for each of the input terms; and

automatically calculating one or more scores that quantify the match between each of the candidate glossary term matches and its respective input term,

wherein the calculation of the one or more scores that quantify the match comprises:

calculating a match score between each of the input terms and each of its identified candidate glossary term matches, as:

a sum of the similarity scores of the fuzzy tokens associated with the respective candidate glossary term matches,

the sum being normalized to a length of the fuzzy tokens associated with the respective candidate glossary term matches, relative to a total length of the input terms, wherein the normalized sum being factored by a ratio between (a) a number of words in each of the candidate glossary term matches whose fuzzy tokens were matched with the automatically comparing step, and (b) a total number of words in each of the candidate glossary term matches.

2. The computer-implemented method of claim 1 , wherein each of the fuzzy tokens consists of a subset of letters of its respective glossary term.

3. The computer-implemented method of claim 1 , wherein the calculation of the similarity score comprises calculating a distance between the respective fuzzy token and its respective word.

4. The computer-implemented method of claim 3 , wherein, in the calculation of the similarity score, a smaller weight is given to vowel omission than to consonant omission.

5. The computer-implemented method of claim 1 , wherein the separating of each of the input terms into the multiple input tokens is based on a location in the respective input term of at least one selected from a group consisting of: a separator character, a white space, and a medial capital.

6. The computer-implemented method of claim 1 , wherein the generated n-grams are of various lengths, from a length of two letters up to a length of the respective input token.

7. The computer-implemented method of claim 1 , the method further comprising:

filtering out the candidate glossary term matches, based on the similarity scores of the fuzzy tokens.

8. A system comprising:

(a) at least one hardware processor; and

(b) a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by said at least one hardware processor to:

obtain multiple glossary terms each comprising one or more words; automatically operate a fuzzy token generator to generate multiple fuzzy tokens from each word of each of the glossary terms;

automatically calculate a similarity score for each of the fuzzy tokens, wherein the similarity score denotes a similarity between the respective fuzzy token and its respective word;

obtain multiple input terms to be matched with the multiple glossary terms; automatically operate a tokenizer to separate each of the input terms into multiple input tokens;

automatically generate multiple n-grams from each of the input tokens;

automatically compare the n-grams with the fuzzy tokens, to output a list of matching n-grams and fuzzy tokens;

based on the list of matching n-grams and fuzzy tokens, automatically identify, from the glossary terms, candidate glossary term matches for each of the input terms; and

automatically calculate one or more scores that quantify the match between each of the candidate glossary term matches and its respective input term,

wherein the calculation of the one or more scores that quantify the match comprises:

calculating a match score between each of the input terms and each of its identified candidate glossary term matches, as:

a sum of the similarity scores of the fuzzy tokens associated with the respective candidate glossary term matches,

the sum being normalized to a length of the fuzzy tokens associated with the respective candidate glossary term matches, relative to a total length of the input terms, wherein the normalized sum being factored by a ratio between (a) a number of words in each of the candidate glossary term matches whose fuzzy tokens were matched with the automatically comparing step, and (b) a total number of words in each of the candidate glossary term matches.

9. The system of claim 8 , wherein each of the fuzzy tokens consists of a subset of letters of its respective glossary term.

10. The system of claim 8 , wherein the calculation of the similarity score comprises calculating a distance between the respective fuzzy token and its respective word.

11. The system of claim 10 , wherein, in the calculation of the similarity score, a smaller weight is given to vowel omission than to consonant omission.

12. The system of claim 8 , wherein the separating of each of the input terms into the multiple input tokens is based on a location in the respective input term of at least one of: a separator character, a white space, and a medial capital.

13. The system of claim 8 , wherein the generated n-grams are of various lengths, from a length of two letters up to a length of the respective input token.

14. The system of claim 8 , further comprising filtering out the candidate glossary term matches, based on the similarity scores of the fuzzy tokens.

15. A computer-implemented method comprising:

obtaining multiple glossary terms each comprising one or more words; automatically operating a fuzzy token generator to generate multiple fuzzy tokens from each word of each of the glossary terms;

automatically calculating a similarity score for each of the fuzzy tokens, wherein the similarity score denotes a similarity between the respective fuzzy token and its respective word;

obtaining multiple input terms to be matched with the multiple glossary terms;

automatically operating a tokenizer to separate each of the input terms into multiple input tokens;

automatically generating multiple n-grams from each of the input tokens;

automatically comparing the n-grams with the fuzzy tokens, to output a list of matching n-grams and fuzzy tokens;

based on the list of matching n-grams and fuzzy tokens, automatically identifying, from the glossary terms, candidate glossary term matches for each of the input terms; and

automatically calculating one or more scores that quantify the match between each of the candidate glossary term matches and its respective input term,

wherein the calculation of the one or more scores that quantify the match comprises:

calculating a match score between each of the input terms and each of its identified candidate glossary term matches, as:

a sum of the similarity scores of the fuzzy tokens associated with the respective candidate glossary term matches,

the sum being normalized to a length of the fuzzy tokens associated with the respective candidate glossary term matches, relative to a total length of the input terms; and

calculating a relation score between each of the input terms and each of its identified candidate glossary term matches, as:

a sum of the similarity scores of the fuzzy tokens associated with the respective candidate glossary term matches,

the sum being normalized to a length of the fuzzy tokens associated with the respective candidate glossary term matches, relative to a total length of the input terms that match the words of each of the respective candidate glossary term matches,

the normalized sum being factored by a ratio between (a) a number of words in each of the candidate glossary term matches whose fuzzy tokens were matched with the automatically comparing step, and (b) a total number of words in each of the candidate glossary term matches.

Assignments (1)

Continuity (1)

References Cited (25)