Multi-service business platform system having entity resolution systems and methods
The disclosure is directed to various ways of improving the functioning of computer systems, information networks, data stores, search engine systems and methods, and other advantages. Among other things, provided herein are methods, systems, components, processes, modules, blocks, circuits, sub-systems, articles, and other elements (collectively referred to in some cases as the “platform” or the “system”) that collectively enable, in one or more datastores (e.g., where each datastore may include one or more databases) and systems, the creation, development, maintenance, and use of a set of custom objects for use in a wide range of activities, including sales activities, marketing activities, service activities, content development activities, and others, as well as improved methods and systems for sales, marketing and services that make use of such entity resolution systems and methods as well as custom objects.
1. A method, comprising:
selecting a quantity and type of neural networks to use for deduplicating objects based upon computing resources and training time;
processing, by a neural network having a type selected for deduplicating the objects, a set of entity feature encodings of entities for deduplication to generate a first reduced vector and a second reduced vector, wherein the neural network is implemented as a vector dimension reducing neural network to produce a reduced dimension entity-specific vector for each entity in a pair that represents feature vectors for a corresponding entity in the pair;
generating representations of likelihoods that entity pairs of the entities are duplicates based upon a dot product of the first reduced vector and the second reduced vector;
training a machine learning model using the likelihoods that entity pairs of the entities are duplicates and p-merge values corresponding to a pair of entities for which a current reduced dimension entity-specific vector is generated by the neural network; and
utilizing the machine learning model to deduplicate the objects within a database, wherein a front-end text encoder, a middle stage trained neural network, and a back-end merge indicator function are selected and used to process the entity pairs for deduplicating the objects, wherein the middle stage trained neural network is replicated to scale the machine learning model for concurrently handling multiple pairs of entities.
2. The method of claim 1 , wherein the utilizing the machine learning model comprises:
matching, by artificial intelligence implemented as a proxy for a deduplication process, strings associated with the objects.
3. The method of claim 1 , wherein the utilizing the machine learning model comprises:
performing, by artificial intelligence implemented as a proxy for a deduplication process, string matching upon the objects.
4. The method of claim 1 , wherein the utilizing the machine learning model comprises:
utilizing the front-end text encoder as a proxy for a deduplication process.
5. The method of claim 1 , wherein the utilizing the machine learning model comprises:
utilizing the middle stage trained neural network as a proxy for a deduplication process.
6. The method of claim 1 , wherein the utilizing the machine learning model comprises:
utilizing the back-end merge indicator function as a proxy for a deduplication process.
7. The method of claim 1 , wherein the neural network is implemented as a dimension-reducing neural network.
8. The method of claim 1 , wherein the p-merge values comprise preconfigured p-merge values derived from string matching of features of the entity pairs and to comparisons of the features of the entity pairs.
9. The method of claim 1 , wherein list segmentation is performed to filter the objects.
10. A system comprising:
a memory comprising machine executable code; and
a processor coupled to the memory, the processor configured to execute the machine executable code to cause the processor to perform operation comprising:
selecting a quantity and type of neural networks to use for deduplicating objects based upon computing resources and training time;
processing, by a neural network having a type selected for deduplicating the objects, a set of entity feature encodings of entities for deduplication to generate a first reduced vector and a second reduced vector, wherein the neural network is implemented as a vector dimension reducing neural network to produce a reduced dimension entity-specific vector for each entity in a pair that represents feature vectors for a corresponding entity in the pair;
generating representations of likelihoods that entity pairs of the entities are duplicates based upon a dot product of the first reduced vector and the second reduced vector;
training a machine learning model using the likelihoods that entity pairs of the entities are duplicates and p-merge values corresponding to a pair of entities for which a current reduced dimension entity-specific vector is generated by the neural network; and
utilizing the machine learning model to deduplicate the objects within a database, wherein a front-end text encoder, a middle stage trained neural network, and a back-end merge indicator function are selected and used to process the entity pairs for deduplicating the objects, wherein the middle stage trained neural network is replicated to scale the machine learning model for concurrently handling multiple pairs of entities.
11. The system of claim 10 , wherein the operations comprise:
matching, by artificial intelligence implemented as a proxy for a deduplication process, strings associated with the objects.
12. The system of claim 10 , wherein the operations comprise:
performing, by artificial intelligence implemented as a proxy for a deduplication process, string matching upon the objects.
13. The system of claim 10 , wherein the operations comprise:
utilizing the front-end text encoder as a proxy for a deduplication process.
14. The system of claim 10 , wherein the operations comprise:
utilizing the middle stage trained neural network as a proxy for a deduplication process.
15. The system of claim 10 , wherein the operations comprise:
utilizing the back-end merge indicator function as a proxy for a deduplication process.
16. A non-transitory machine-readable storage medium comprising instructions that when executed by a machine, causes the machine to perform operations comprising:
selecting a quantity and type of neural networks to use for deduplicating objects based upon computing resources and training time;
processing, by a neural network having a type selected for deduplicating the objects, a set of entity feature encodings of entities for deduplication to generate a first reduced vector and a second reduced vector, wherein the neural network is implemented as a vector dimension reducing neural network to produce a reduced dimension entity-specific vector for each entity in a pair that represents feature vectors for a corresponding entity in the pair;
generating representations of likelihoods that entity pairs of the entities are duplicates based upon a dot product of the first reduced vector and the second reduced vector;
training a machine learning model using the likelihoods that entity pairs of the entities are duplicates and p-merge values corresponding to a pair of entities for which a current reduced dimension entity-specific vector is generated by the neural network; and
utilizing the machine learning model to deduplicate the objects within a database, wherein a front-end text encoder, a middle stage trained neural network, and a back-end merge indicator function are selected and used to process the entity pairs for deduplicating the objects, wherein the middle stage trained neural network is replicated to scale the machine learning model for concurrently handling multiple pairs of entities.
17. The non-transitory machine-readable storage medium of claim 16 , wherein the operations comprise:
matching, by artificial intelligence implemented as a proxy for a deduplication process, strings associated with the objects.
18. The non-transitory machine-readable storage medium of claim 16 , wherein the operations comprise:
performing, by artificial intelligence implemented as a proxy for a deduplication process, string matching upon the objects.
19. The non-transitory machine-readable storage medium of claim 16 , wherein the operations comprise:
utilizing the front-end text encoder as a proxy for a deduplication process threshold.
20. The non-transitory machine-readable storage medium of claim 16 , wherein the operations comprise:
utilizing the middle stage trained neural network as a proxy for a deduplication process.