Systems and methods for data correlation and artifact matching in identity management artificial intelligence systems

1. An identity management system, comprising:

a processor;

a non-transitory, computer-readable storage medium, including computer instructions for:

obtaining identity management data associated with a plurality of source systems in a distributed enterprise computing environment, the identity management data comprising data on a set of identity management artifacts utilized in identity management in the distributed enterprise computing environment, wherein the plurality of source systems include a non-authoritative source system and an authoritative source system and the identity management data comprises account data on accounts from the non-authoritative source system and identity data on identities from the authoritative source system;

determining a first set of columns associated with a schema of the non-authoritative source system;

determining a second set of columns associated with a schema of the authoritative source system;

forming a set of feature pairs specific to the non-authoritative source system and the authoritative source system wherein each feature pair of the set of feature pairs comprises a first column from the first set of columns associated with the schema of the non-authoritative source system and a second column from the second set of columns associated with the schema of the authoritative source system by correlating the first set of columns with the second set of columns to determine the set of feature pairs;

generating feature values for each of the feature pairs for each of a set of account-identity pairs, where each account-identity pair comprises a first account of the accounts of the account data from the non-authoritative source system and a first identity of the identities of the identity data from the authoritative source system, and generating a feature value for a feature pair is based on a first value for the first column of the feature pair associated with the first account and a second value for the second column of the feature pair associated with the first identity;

obtaining a training set of account-identity pairs and associated feature values;

training a machine learning (ML) model specific to the non-authoritative source system and the authoritative source system based on the training set;

filtering the set of account-identity pairs to select one or more account-identity pairs for which predictions are to be determined, wherein the filtering is based on the feature values for a set of screening feature pairs for each of the set of account-identity pairs and the set of screening feature pairs are a top number of feature pairs whose first column and second column have highest similarity values;

generating, by the machine learning model, predictions for the selected one or more account-identity pairs, wherein a prediction for an account-identity pair is based on the feature values associated with that account-identity pair and if the prediction is over a threshold the account of the account-identity pair is associated with the identity of the account-identity pair.

2. The identity management system of claim 1 , wherein correlating the first set of columns with the second set of columns to determine the set of feature pairs comprises:

determining a similarity value between each of the first set of columns and each of the second set of columns based on first values for the first column across all the accounts of the account data and second values for the second column across all identities of the identity data; and

selecting the feature pairs based on the similarity values.

3. The identity management system of claim 1 , wherein the instructions further comprise instructions for:

receiving a request for an interpretation of the prediction for an identified account-identity pair;

determining a top set of features pairs that resulted in the prediction based on the ML model; and

returning the top set of features to a user.

4. The identity management system of claim 3 , wherein determining the top set of feature pairs comprise querying the ML model to build a local model for the account-identity pair for which the prediction was generated using the ML model and determining the top set of feature pairs from the local model.

5. A method, comprising:

obtaining identity management data associated with a plurality of source systems in a distributed enterprise computing environment, the identity management data comprising data on a set of identity management artifacts utilized in identity management in the distributed enterprise computing environment, wherein the plurality of source systems include a non-authoritative source system and an authoritative source system and the identity management data comprises account data on accounts from the non-authoritative source system and identity data on identities from the authoritative source system;

determining a first set of columns associated with a schema of the non-authoritative source system;

determining a second set of columns associated with a schema of the authoritative source system;

forming a set of feature pairs specific to the non-authoritative source system and the authoritative source system wherein each feature pair of the set of feature pairs comprises a first column from the first set of columns associated with the schema of the non-authoritative source system and a second column from the second set of columns associated with the schema of the authoritative source system by correlating the first set of columns with the second set of columns to determine the set of feature pairs;

generating feature values for each of the feature pairs for each of a set of account-identity pairs, where each account-identity pair comprises a first account of the accounts of the account data from the non-authoritative source system and a first identity of the identities of the identity data from the authoritative source system, and generating a feature value for a feature pair is based on a first value for the first column of the feature pair associated with the first account and a second value for the second column of the feature pair associated with the first identity;

obtaining a training set of account-identity pairs and associated feature values;

training a machine learning (ML) model specific to the non-authoritative source system and the authoritative source system based on the training set;

filtering the set of account-identity pairs to select one or more account-identity pairs for which predictions are to be determined, wherein the filtering is based on the feature values for a set of screening feature pairs for each of the set of account-identity pairs and the set of screening feature pairs are a top number of feature pairs whose first column and second column have highest similarity values;

generating, by the machine learning model, predictions for the selected one or more account-identity pairs, wherein a prediction for an account-identity pair is based on the feature values associated with that account-identity pair and if the prediction is over a threshold the account of the account-identity pair is associated with the identity of the account-identity pair.

6. The method of claim 5 , wherein correlating the first set of columns with the second set of columns to determine the set of feature pairs comprises:

determining a similarity value between each of the first set of columns and each of the second set of columns based on first values for the first column across all the accounts of the account data and second values for the second column across all identities of the identity data; and

selecting the feature pairs based on the similarity values.

7. The method of claim 5 , further comprising:

receiving a request for an interpretation of the prediction for an identified account-identity pair;

determining a top set of features pairs that resulted in the prediction based on the ML model; and

returning the top set of features to a user.

8. The method of claim 7 , wherein determining the top set of feature pairs comprise querying the ML model to build a local model for the account-identity pair for which the prediction was generated using the ML model and determining the top set of feature pairs from the local model.

9. A non-transitory computer readable medium, comprising instructions for:

obtaining identity management data associated with a plurality of source systems in a distributed enterprise computing environment, the identity management data comprising data on a set of identity management artifacts utilized in identity management in the distributed enterprise computing environment, wherein the plurality of source systems include a non-authoritative source system and an authoritative source system and the identity management data comprises account data on accounts from the non-authoritative source system and identity data on identities from the authoritative source system;

determining a first set of columns associated with a schema of the non-authoritative source system;

determining a second set of columns associated with a schema of the authoritative source system;

forming a set of feature pairs specific to the non-authoritative source system and the authoritative source system wherein each feature pair of the set of feature pairs comprises a first column from the first set of columns associated with the schema of the non-authoritative source system and a second column from the second set of columns associated with the schema of the authoritative source system by correlating the first set of columns with the second set of columns to determine the set of feature pairs;

generating feature values for each of the feature pairs for each of a set of account-identity pairs, where each account-identity pair comprises a first account of the accounts of the account data from the non-authoritative source system and a first identity of the identities of the identity data from the authoritative source system, and generating a feature value for a feature pair is based on a first value for the first column of the feature pair associated with the first account and a second value for the second column of the feature pair associated with the first identity;

obtaining a training set of account-identity pairs and associated feature values;

training a machine learning (ML) model specific to the non-authoritative source system and the authoritative source system based on the training set;

filtering the set of account-identity pairs to select one or more account-identity pairs for which predictions are to be determined, wherein the filtering is based on the feature values for a set of screening feature pairs for each of the set of account-identity pairs and the set of screening feature pairs are a top number of feature pairs whose first column and second column have highest similarity values;

generating, by the machine learning model, predictions for the selected one or more account-identity pairs, wherein a prediction for an account-identity pair is based on the feature values associated with that account-identity pair and if the prediction is over a threshold the account of the account-identity pair is associated with the identity of the account-identity pair.

10. The non-transitory computer readable medium of claim 9 , wherein correlating the first set of columns with the second set of columns to determine the set of feature pairs comprises:

determining a similarity value between each of the first set of columns and each of the second set of columns based on first values for the first column across all the accounts of the account data and second values for the second column across all identities of the identity data; and

selecting the feature pairs based on the similarity values.

11. The non-transitory computer readable medium of claim 9 , further comprising instructions for:

receiving a request for an interpretation of the prediction for an identified account-identity pair;

determining a top set of features pairs that resulted in the prediction based on the ML model; and

returning the top set of features to a user.

12. The non-transitory computer readable medium of claim 11 , wherein determining the top set of feature pairs comprise querying the ML model to build a local model for the account-identity pair for which the prediction was generated using the ML model and determining the top set of feature pairs from the local model.