Entity linking and filtering using efficient search tree and machine learning representations
Methods, systems, and computer-readable storage media for a ML system that reduces a number of target items from consideration as potential matches to a query item using token embeddings and a search tree.
1. A computer-implemented method for matching a query item to one or more target items using machine learning (ML) models, the method being executed by one or more processors and comprising:
receiving query item text associated with a query item that is to be matched to one or more target items of a superset of target items, the query item text comprising one or more query item tokens;
prior to using a second ML model during inference to match the query item to one or more target items of the superset of target items, providing a set of target items from the superset of target items by:
for a first query item token of the query item text:
determining, by a first ML model, a first query item token embedding,
comparing the first query item token embedding to target item token embeddings of target items tokens included within a search space to identify at least one target item token that is sufficiently similar to the first query item token, and
associating the first query item token with a revised search space within a tracker, the tracker comprising an array data structure that is initialized with a set of null values, associating the first query item token with the revised search space within the tracker comprises replacing a null value with a search space index indicating where the first query item token was found in the search space, and wherein the revised search space is provided in a queue of search spaces, a length of the queue being defined by a window parameter;
determining a set of matched item tokens based on the tracker, the set of matched item tokens indicating one of a match and a partial match between a query item token and a target item token, and further based on the length between the items being within a window size of the window parameter;
defining the set of target items from the set of matched item tokens, a number of target items in the set of target items being less than a number of target items in the superset of target items;
determining that no target item tokens represented in the revised search space is sufficiently similar to a second query item token, and in response, comparing a second query item token embedding to target item token embeddings of target items tokens included within an alternative search space present in the queue; and
executing inference to match the query item to one or more target items in the set of target items by processing the query item and target items of the set of target items through the second ML model that outputs inference results, the inference results indicating a match between the query item and at least one target item in the set of target items.
2. The method of claim 1 , wherein the search space is defined within a search tree comprising a set of nodes, each node representative of a respective target item token in a set of target item tokens.
3. The method of claim 1 , wherein the revised search space comprises a search sub-space of the search space.
4. The method of claim 1 , further comprising, for the second query item token of the query item text:
determining, by the first ML model, the second query item token embedding; and
comparing the second query item token embedding to target item token embeddings of target items tokens included within the revised search space.
5. The method of claim 1 , wherein, during a training phase, the first ML model is fine-tuned based on sets of perturbations, each set of perturbations corresponding to an item token.
6. A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for matching a query item to one or more target items using machine learning (ML) models, the operations comprising:
receiving query item text associated with a query item that is to be matched to one or more target items of a superset of target items, the query item text comprising one or more query item tokens;
prior to using a second ML model during inference to match the query item to one or more target items of the superset of target items, providing a set of target items from the superset of target items by:
for a first query item token of the query item text:
determining, by a first ML model, a first query item token embedding,
comparing the first query item token embedding to target item token embeddings of target items tokens included within a search space to identify at least one target item token that is sufficiently similar to the first query item token, and
associating the first query item token with a revised search space within a tracker, the tracker comprising an array data structure that is initialized with a set of null values, associating the first query item token with the revised search space within the tracker comprises replacing a null value with a search space index indicating where the first query item token was found in the search space, and wherein the revised search space is provided in a queue of search spaces, a length of the queue being defined by a window parameter;
determining a set of matched item tokens based on the tracker, the set of matched item tokens indicating one of a match and a partial match between a query item token and a target item token, and further based on the length between the items being within a window size of the window parameter;
defining the set of target items from the set of matched item tokens, a number of target items in the set of target items being less than a number of target items in the superset of target items;
determining that no target item tokens represented in the revised search space is sufficiently similar to a second query item token, and in response, comparing a second query item token embedding to target item token embeddings of target items tokens included within an alternative search space present in the queue; and
executing inference to match the query item to one or more target items in the set of target items by processing the query item and target items of the set of target items through the second ML model that outputs inference results, the inference results indicating a match between the query item and at least one target item in the set of target items.
7. The non-transitory computer-readable storage medium of claim 6 , wherein the search space is defined within a search tree comprising a set of nodes, each node representative of a respective target item token in a set of target item tokens.
8. The non-transitory computer-readable storage medium of claim 6 , wherein the revised search space comprises a search sub-space of the search space.
9. The non-transitory computer-readable storage medium of claim 6 , wherein operations further comprise, for a second query item token of the query item text:
determining, by the first ML model, the second query item token embedding; and
comparing the second query item token embedding to target item token embeddings of target items tokens included within the revised search space.
10. The non-transitory computer-readable storage medium of claim 6 , wherein, during a training phase, the first ML model is fine-tuned based on sets of perturbations, each set of perturbations corresponding to an item token.
11. A system, comprising:
a computing device comprising one or more processors; and
a non-transitory computer-readable storage device coupled to the computing device and having instructions stored thereon which, when executed by the computing device, cause the computing device to perform operations for matching a query item to one or more target items using machine learning (ML) models, the operations comprising:
receiving query item text associated with a query item that is to be matched to one or more target items of a superset of target items, the query item text comprising one or more query item tokens;
prior to using a second ML model during inference to match the query item to one or more target items of the superset of target items, providing a set of target items from the superset of target items by:
for a first query item token of the query item text:
determining, by a first ML model, a first query item token embedding,
comparing the first query item token embedding to target item token embeddings of target items tokens included within a search space to identify at least one target item token that is sufficiently similar to the first query item token, and
associating the first query item token with a revised search space within a tracker, the tracker comprising an array data structure that is initialized with a set of null values, associating the first query item token with the revised search space within the tracker comprises replacing a null value with a search space index indicating where the first query item token was found in the search space, and wherein the revised search space is provided in a queue of search spaces, a length of the queue being defined by a window parameter;
determining a set of matched item tokens based on the tracker, the set of matched item tokens indicating one of a match and a partial match between a query item token and a target item token, and further based on the length between the items being within a window size of the window parameter;
defining the set of target items from the set of matched item tokens, a number of target items in the set of target items being less than a number of target items in the superset of target items;
determining that no target item tokens represented in the revised search space is sufficiently similar to a second query item token, and in response, comparing a second query item token embedding to target item token embeddings of target items tokens included within an alternative search space present in the queue; and
executing inference to match the query item to one or more target items in the set of target items by processing the query item and target items of the set of target items through the second ML model that outputs inference results, the inference results indicating a match between the query item and at least one target item in the set of target items.
12. The system of claim 11 , wherein the search space is defined within a search tree comprising a set of nodes, each node representative of a respective target item token in a set of target item tokens.
13. The system of claim 11 , wherein the revised search space comprises a search sub-space of the search space.
14. The system of claim 11 , wherein operations further comprise, for a second query item token of the query item text:
determining, by the first ML model, the second query item token embedding; and
comparing the second query item token embedding to target item token embeddings of target items tokens included within the revised search space.