Systems and methods for multi-purpose data management

1 . A data management system comprising:

one or more memory devices storing processor executable instructions; and

one or more processors configured to execute the instructions to cause the data management system to perform operations, the operations comprising:

receiving at least two sets of data and a data management task request, wherein the at least two sets of data include a set of entities;

determining a location of the set of entities in a representative space, wherein the location in the representative space is determined based on a representative structure of the set of entities;

determining a set of corresponding representative entity pairs from the at least two sets of data, wherein entities of the set of corresponding representative entity pairs are determined to be closer in the representative space based on an angle between vector representations of the entities of the set of corresponding representative entity pairs;

analyzing the set of corresponding representative entity pairs to identify a set of candidate pairs, wherein the set of candidate pairs includes entity pairs of the corresponding representative entity pairs that are determined to be most similar entity pairs, wherein the most similar entity pairs are determined according to a closeness of location in the representative space of entities of the set of corresponding representative entity pairs;

determining matched entity pairs of the set of candidate pairs, wherein a first machine learning model is trained using the set of candidate pairs by applying labels; and

utilizing the matched entity pairs to perform the requested data management task.

2 . The data management system of claim 1 , wherein determining the location of the set of entities in the representative space further comprises:

pre-training a second model using a contrastive learning technique, wherein the contrastive learning technique includes a contrastive objective and takes as input the at least two sets of data; and

determining the representative structure of the set of entities using the second model.

3 . The data management system of claim 2 , wherein the second model is a convolution neural network.

4 . The data management system of claim 2 , wherein pre-training the second model using a contrastive learning technique further comprises:

generating training data for the second model using an augmentation operator based on the requested data management task, wherein the augmentation operator generates variants of entities in the at least two sets of data;

training the second model to determine similarity or dissimilarity between an entity of the set of entities and a variant of the entity, wherein training the second model further comprises:

encoding the entity and the variant of the entity using an embedding model; and

projecting the encoded entity and the encoded variant in a vector space.

5 . The data management system of claim 1 , wherein the entities of the corresponding representative entity pairs are determined to be closer in the representative space based on an angle between vector representations of the entities of the corresponding representative entity pairs when the angle is more than a threshold value.

6 . The data management system of claim 1 , wherein the set of candidate pairs is used to train the first machine learning model by applying labels to a subset of the set of corresponding representative entity pairs, wherein the subset of the set of corresponding representative entity pairs includes entity pairs with similarity between vector representations above a positive threshold value or below a negative threshold value.

7 . The data management system of claim 6 , wherein applying labels to the subset of the set of corresponding representative entity pairs further comprises:

applying a positive value label to a first entity pair with entities that are determined to be closer to each other, wherein entities in the first entity pair are similar to each other; and

applying a negative value label to a second entity pair with entities that are determined to be farther from each other, wherein entities in the second entity are dissimilar from each other.

8 . The data management system of claim 7 , wherein applying the positive value label to the first entity pair further comprises:

varying an amount of negative value depending on an amount of distance between entities in the first entity pair.

9 . The data management system of claim 7 , wherein applying the negative value label to the second entity pair further comprises:

varying an amount of positive value depending on an amount of distance between entities in the second entity pair.

10 . The data management system of claim 1 wherein a data management task request includes one of: data integration, data cleanup, or data discovery.

11 . The data management system of claim 1 , wherein determining matched entity pairs of the set of candidate pairs includes binary classification of at least one entity pair as matched or unmatched.

12 . The data management system of claim 1 , wherein determining matched entity pairs of the set of candidate pairs includes determining a level of match between entities of at least one entity pair.

13 . The data management system of claim 1 , wherein the candidate pairs are used to train the first machine learning model that includes a linear layer based on the requested data management task.

14 . A method for performing data management tasks on entities utilizing a data management system, the method comprising:

receiving at least two sets of data and a data management task request, wherein the at least two sets of data include a set of entities;

determining a location of the set of entities in a representative space, wherein the location in the representative space is determined based on a representative structure of the set of entities;

determining, a set of corresponding representative entity pairs from the at least two sets of data, wherein entities in each of the set of corresponding representative entity pairs are determined to be closer in the representative space based on an angle between vector representations of the entities of the set of corresponding representative entity pairs;

analyzing the set of corresponding representative entity pairs to identify a set of candidate pairs, wherein the set of candidate pairs includes entity pairs of the corresponding representative entity pairs that are determined to be most similar entity pairs, wherein the most similar entity pairs are determined according to a closeness of location in the representative space of entities of the set of corresponding representative entity pairs;

determining matched entity pairs of the set of candidate pairs, wherein a first machine learning model is trained using the set of candidate pairs by applying labels; and

utilizing the matched entity pairs to perform the requested data management task.

15 . The method of claim 14 , wherein determining the location of the set of entities in the representative space further comprises:

pre-training a second model using a contrastive learning technique, wherein the contrastive learning technique includes a contrastive objective and takes as input the at least two sets of data; and

determining the representative structure of the set of entities using the second model.

16 . The method of claim 14 , wherein the entities of the corresponding representative entity pairs are determined to be closer in the representative space based on an angle between vector representations of the entities of the corresponding representative entity pairs when the angle is more than a threshold value.

17 . The method of claim 14 , wherein the set of candidate pairs is used to train the first machine learning model by applying labels to a subset of the set of corresponding representative entity pairs; wherein the subset of the set of corresponding representative entity pairs includes entity pairs with similarity between vector representations above a positive threshold value or below a negative threshold value.

18 . The method of claim 17 , wherein applying labels to the subset of the set of corresponding representative entity pairs further comprises:

applying a positive value label to a first entity pair with entities that are determined to be closer to each other, wherein entities in the first entity pair are similar to each other; and

applying a negative value label to a second entity pair with entities that are determined to be farther from each other, wherein entities in the second entity pair are dissimilar from each other.

19 . The method of claim 18 , wherein applying the positive value label to the first entity pair further comprises:

varying an amount of negative value depending on amount of distance between entities in the first entity pair.

20 . A non-transitory computer readable medium including instructions that are executable by one or more processors to cause a system to perform a method performing data management task on entities, the method comprising:

receiving at least two sets of data and a data management task request, wherein the at least two sets of data include a set of entities;

determining a location of the set of entities in a representative space, wherein the location in the representative space is determined based on a representative structure of the set of entities;

determining a set of corresponding representative entity pairs from the at least two sets of data, wherein entities of the set of corresponding representative entity pairs are determined to be closer in the representative space based on an angle between vector representations of the entities of the set of corresponding representative entity pairs;

analyzing the set of corresponding representative entity pairs to identify a set of candidate pairs, wherein the set of candidate pairs includes entity pairs of the corresponding representative entity pairs that are determined to be most similar entity pairs, wherein the most similar entity pairs are determined according to a closeness of location in the representative space of entities of the set of corresponding representative entity pairs;

determining matched entity pairs of the set of candidate pairs, wherein a first machine learning model is trained using the set of candidate pairs by applying labels; and

utilizing the matched entity pairs to perform the requested data management task.