Machine learning enabled real time query handling system and method

1 . A computing device comprising:

a communications module;

a processor coupled with the communications module; and

a memory coupled to the processor and storing processor-executable instructions which, when executed by the processor, configure the computing device to:

perform word tokenization on corporate identifiers in a first identifier set to obtain a first set of tokens wherein the first identifier set is based on a corporate structure file which includes one or more of a company name, a country of incorporation and an address;

perform word tokenization on identifiers in a second identifier set to obtain a second set of tokens;

obtain a weighting value for each of a plurality of tokens in the first set of tokens and the second set of tokens based on a frequency of occurrence while deemphasizing the weighting value of corporate suffix tokens indicating legal entity form;

for each corporate identifier of a plurality of corporate identifiers in the first identifier set and of a plurality of identifiers in the second identifier set, obtain embeddings for the identifier based on embeddings of tokens obtained by performing word tokenization on that identifier and based on the weighting value for such tokens;

obtain similarity metrics for corporate identifiers in the first identifier set and identifiers in the second identifier set based on the embeddings for such identifiers;

store a similarity profile based on the similarity metrics for corporate identifiers in the first identifier set, the similarity profile including data mapping one or more corporate identifiers in the first identifier set to one or more identifiers in the second identifier set;

receive a query; and

generate a user interface based on a selected similarity profile identified based on the query.

2 . The computing device of claim 1 , wherein the instructions further configure the computing device to:

prior to performing word tokenization on the corporate identifiers in the first identifier set, pre-processing the first identifier set to standardize the corporate identifiers in the first identifier set; and

prior to performing word tokenization on the identifiers in the second identifier set, pre-processing the second identifier set to standardize the identifiers in the second identifier set.

3 . The computing device of claim 2 , wherein pre-processing to standardize includes one or more of:

converting characters to a particular letter case; and

removing one or more characters specified in a character blacklist.

4 . The computing device of claim 1 , wherein obtain a weighting value includes obtaining a weighting value based on an exponentially decaying weighting function.

5 . The computing device of claim 4 , wherein the exponentially decaying weighting function is configured to de-emphasize tokens having a higher frequency of occurrence.

6 . The computing device of claim 5 , wherein the exponentially decaying weighting function uses a hyperparameter to de-emphasize tokens having a higher frequency of occurrence.

7 . The computing device of claim 6 , wherein obtain a weighting value includes performing a grid search.

8 . The computing device of claim 1 , wherein obtaining embeddings for the identifier based on embeddings of tokens obtained by performing word tokenization on that identifier and based on the weighting value for such tokens includes:

generating embeddings of each token in the plurality of tokens in the first set of tokens and the second set of tokens, the embeddings being a vectorized representation of the tokens; and

generating the embeddings for one of the identifiers based on the embeddings for the tokens represented by that one of the identifiers.

9 . The computing device of claim 1 , wherein the query is an input of a corporate identifier in the first identifier set and wherein the user interface is generated based a corresponding identifier in the second identifier set, the corresponding identifier identified based on the selected similarity profile.

10 . The computing device of claim 1 , wherein the instructions further configure the computing device to:

perform word tokenization on identifiers in a third identifier set to obtain a third set of tokens;

obtain a weighting value for each of a plurality of tokens in the third set of based on a frequency of occurrence;

for each of a plurality of identifiers in the third identifier set, obtain embeddings for the identifier based on embeddings of tokens obtained by performing word tokenization on that identifier and based on the weighting value for such tokens;

obtain similarity metrics for identifiers in the third identifier set and corporate identifiers in the first identifier set based on the embeddings for such identifiers; and

compare the similarity metrics obtained for the corporate identifiers in the first identifier set and identifiers in the second identifier set to similarity metrics obtained for the identifiers in the third identifier set and corporate identifiers in the first identifier set to map identifiers in the third identifier set that map to identifiers in the second identifier set.

11 . The computing device of claim 1 , wherein obtain a weighting value for each of a plurality of tokens in the first set of tokens and the second set of tokens based on a frequency of occurrence comprises:

combining the first set of tokens and the second set of tokens to obtain a dictionary of tokens; and

determining the frequency of occurrence for each of the tokens in the dictionary of tokens, the frequency of occurrence indicating the frequency of occurrence in the first identifier set and the second identifier set.

12 . A computer-implemented method comprising:

performing word tokenization on corporate identifiers in a first identifier set to obtain a first set of tokens wherein the first identifier set is based on a corporate structure file which includes one or more of a company name, a country of incorporation and an address;

performing word tokenization on identifiers in a second identifier set to obtain a second set of tokens;

obtain a weighting value for each of a plurality of tokens in the first set of tokens and the second set of tokens based on a frequency of occurrence while deemphasizing the weighting value of corporate suffix tokens indicating legal entity form;

for each corporate identifier of a plurality of corporate identifiers in the first identifier set and of a plurality of identifiers in the second identifier set, obtaining embeddings for the identifier based on embeddings of tokens obtained by performing word tokenization on that identifier and based on the weighting value for such tokens;

obtaining similarity metrics for corporate identifiers in the first identifier set and identifiers in the second identifier set based on the embeddings for such identifiers;

storing a similarity profile based on the similarity metrics for corporate identifiers in the first identifier set, the similarity profile including data mapping one or more corporate identifiers in the first identifier set to one or more identifiers in the second identifier set;

receiving a query; and

generating a user interface based on a selected similarity profile identified based on the query.

13 . The method of claim 12 , further comprising:

prior to performing word tokenization on the corporate identifiers in the first identifier set, pre-processing the first identifier set to standardize the corporate identifiers in the first identifier set; and

prior to performing word tokenization on the identifiers in the second identifier set, pre-processing the second identifier set to standardize the identifiers in the second identifier set.

14 . The method of claim 13 , wherein pre-processing to standardize includes one or more of:

converting characters to a particular letter case; and

removing one or more characters specified in a character blacklist.

15 . The method of claim 12 , wherein obtaining a weighting value includes obtaining a weighting value based on an exponentially decaying weighting function.

16 . The method of claim 15 , wherein the exponentially decaying weighting function is configured to de-emphasize tokens having a higher frequency of occurrence.

17 . The method of claim 16 , wherein the exponentially decaying weighting function uses a hyperparameter to de-emphasize tokens having a higher frequency of occurrence.

18 . The method of claim 17 , wherein obtain a weighting value includes performing a grid search.

19 . The method of claim 12 wherein obtaining embeddings for the identifier based on embeddings of tokens obtained by performing word tokenization on that identifier and based on the weighting value for such tokens includes:

generating embeddings of each token in the plurality of tokens in the first set of tokens and the second set of tokens, the embeddings being a vectorized representation of the tokens; and

generating the embeddings for one of the identifiers based on the embeddings for the tokens represented by that one of the identifiers.

20 . The computing device of claim 11 , wherein the instructions further configure the computing device to: perform clustering, a machine learning technique, on the tokens in the dictionary of tokens.