Machine learning embeddings for evolving category sets
In some embodiments, a computing system may generate a prediction related to a new category (not included in a set of categories) using a machine learning model trained on a set of embeddings corresponding to the category set. As an example, the computing system may generate a set of hashes such that each hash of the hash set is mapped to an embedding of the embedding set. When a new category added to the category set, the computing system may generate a given hash for the new category and identify a first hash of the hash set that matches the given hash. Based on identifying the first hash as a matching hash, the computing system may use an existing embedding (e.g., mapped to the first hash) with the machine learning model in connection with the new category, thereby avoiding a need to add a new embedding to the embedding set.
1 . A system for reducing computing resources for training machine learning models by generating a category embedding set comprising categories to which new categories are mapped, the system comprising:
one or more processors programmed with instructions that, when executed by the one or more processors, cause operations comprising:
storing a set of embeddings and a set of hashes mapped to a dimensional space configured to have a potential number of hashes that is less than a number of vocabulary words available in a vocabulary collection, wherein each embedding in the set of embeddings corresponds to a vocabulary word of a set of vocabulary words of the vocabulary collection, and each hash of the set of hashes corresponds to a vocabulary word in the set of vocabulary words;
based on a hash for an other vocabulary word being a first hash corresponding to a first vocabulary word that is different from the other vocabulary word and is in the set of vocabulary words, updating a mapping of a first embedding, corresponding to the first vocabulary word, to also correspond to the other vocabulary word, despite the other vocabulary word being different from the first vocabulary word, wherein the other vocabulary word is not in and not added to the set of vocabulary words; and
in connection with the updating of the mapping of the first embedding to also correspond to the other vocabulary word, processing an input comprising the other vocabulary word by using the first embedding, in lieu of an embedding of the input comprising the other vocabulary word, with a machine learning model to generate output related to the input.
2 . The system of claim 1 , wherein processing the input comprising the other vocabulary word comprises:
generating the embedding of the input comprising the other vocabulary word; and
returning the first embedding for use with the machine learning model, in lieu of the embedding of the input, based on a hash for the input comprising the other vocabulary word being the first hash corresponding to the first vocabulary word different from the other vocabulary word.
3 . The system of claim 2 , wherein the embedding of the input comprises a first portion corresponding to a name of the vocabulary word and a second portion corresponding to other data associated with the vocabulary word.
4 . The system of claim 1 , wherein a total number of hashes of the set of hashes does not exceed a threshold percentage, less than 100%, of a total number of vocabulary words in the set of vocabulary words.
5 . A method comprising:
storing a set of embeddings and a set of hashes mapped to a dimensional space configured to have a potential number of hashes that is less than a number of categories available in a category collection, wherein each embedding in the set of embeddings corresponds to a category of a set of categories of the category collection, and each hash of the set of hashes corresponds to a category in the set of categories;
based on a hash for an other category not in the set of categories being a first hash corresponding to a first category that is different from the other category and in the set of categories, updating a mapping of a first embedding, corresponding to the first category, to also correspond to the other category; and
processing an input comprising the other category by using the first embedding, in lieu of an embedding the input comprising the other category, with a machine learning model to generate output related to the input.
6 . The method of claim 5 , further comprising:
training a second machine learning model to determine mappings between categories and the set of embeddings, wherein inputting category data into the second machine learning model causes output of a classification indicating which embeddings of the set of embeddings are to be used for corresponding categories.
7 . The method of claim 5 , further comprising:
generating, via a second machine learning model, a second embedding for the first category that is different from the first embedding;
determining that a distance metric associated with the first embedding and the second embedding does not satisfy a threshold distance; and
before the updating of the mapping of the first embedding to also correspond to the other vocabulary word, updating the mapping of the first embedding to correspond to the first category based on the determination that the distance metric associated with the first embedding and the second embedding does not satisfy the threshold distance.
8 . The method of claim 5 , wherein a total number of hashes of the set of hashes does not exceed a threshold percentage, less than 100%, of a total number of categories in the set of categories.
9 . The method of claim 5 , wherein processing the input comprises:
generating the embedding of the input comprising the other category; and
returning the first embedding for use with the machine learning model, in lieu of the embedding of the input, based on a hash for the input comprising the other category being the first hash corresponding to the first category different from the other category.
10 . The method of claim 9 , wherein the embedding of the input comprising the other category comprises a first portion corresponding to a name of the other category and a second portion corresponding to a code associated with the other category.
11 . The method of claim 5 , wherein the hash for the input comprises a first portion associated with a tokenized name of the other category and a second portion associated with a tokenized description of the other category.
12 . The method of claim 5 , wherein processing the input comprises processing the input comprising an other product name and description by using the first embedding corresponding to a first product name and description, in lieu of the embedding of the input, with the machine learning model to generate output related to the input, the first product name and description being different from the other product name and description.
13 . One or more non-transitory, computer-readable media comprising instructions that when executed by one or more processors, cause operations comprising:
storing a set of embeddings and a set of hashes, wherein each embedding in the set of embeddings corresponds to a category of an set of categories, and each hash of the set of hashes corresponds to a category in the set of categories;
based on a hash for an other category being a first hash corresponding to a first category that is different from the other category and in the set of categories, updating a mapping of a first embedding, corresponding to the first category, to also correspond to the other category, wherein the other category is not in the set of categories; and
in connection with the updating of the mapping of the first embedding to also correspond to the other category, processing an input comprising the other category by using the first embedding, in lieu of an embedding of the input comprising the other category, with a machine learning model to generate output related to the input.
14 . The media of claim 13 , wherein the set of hashes are mapped to a dimensional space configured to have a potential number of hashes that is less than a number of categories available in a category collection.
15 . The media of claim 13 , the operations further comprising:
generating, via a second machine learning model, a second embedding for the first category that is different from the first embedding; and
before the updating of the mapping of the first embedding to also correspond to the other category, updating the mapping of the first embedding to correspond to the first category based on a distance metric associated with the first embedding and the second embedding being less than a threshold distance.
16 . The media of claim 13 , wherein a total number of hashes of the set of hashes does not exceed a threshold percentage, less than 100%, of a total number of categories in the set of categories.
17 . The media of claim 13 , wherein processing the input comprises:
generating the embedding of the input comprising the other category; and
returning the first embedding for use with the machine learning model, in lieu of the embedding of the input, based on a hash for the input comprising the other category being the first hash corresponding to the first category different from the other category.
18 . The media of claim 17 , wherein the embedding of the input comprising the other category comprises a first portion corresponding to a name of the other category and a second portion corresponding to a code associated with the other category.
19 . The media of claim 13 , wherein the hash for the input comprises a first portion associated with a tokenized name of the other category and a second portion associated with a tokenized description of the other category.
20 . The media of claim 13 , wherein processing the input comprises processing the input comprising an other product name and description by using the first embedding corresponding to a first product name and description, in lieu of the embedding of the input, with the machine learning model to generate output related to the input, the first product name and description being different from the other product name and description.