Determining emebedding vectors for an unmapped content item using embedding inferenece
Systems and methods are presented for inferring an embedding vector of an item of a first type into the embedding space. Upon receiving a first time for which there is no embedding vector, documents of a document corpus that include (co-occurrence) both the received item and other items of the same type are identified. Of those other items that have embedding vectors, those embedding vectors are retrieved and averaged. The resulting averaged embedding vector is established as an inferred embedding vector for the received item.
1 . A computer-implemented method for inferring an embedding vector for an unmapped target content item of a target content type, the method comprising:
accessing document corpus information, the document corpus information describing a plurality of documents that include content items of a plurality of content types, including content items of the target content type;
identifying a set of documents of the document corpus that include the unmapped target content item;
identifying a set of content items of the target content type from the identified set of documents;
accessing embedding vectors of the identified set of content items;
for each document of the identified set of documents:
determining a document subset of content items of the set of content items of the target content type that are included in a current document; and
averaging the embedding vectors of the content items of the document subset of content items, resulting in an averaged document embedding vector for the current document;
averaging the averaged document embedding vectors to generate an averaged embedding vector; and
associating the averaged embedding vector with the unmapped target content item as an inferred embedding vector representing the unmapped target content item.
2 . The computer-implemented method of claim 1 , further comprising:
for each document of the identified set of documents:
determining a frequency of occurrence of each content item of the document subset of content items in the current document; and
averaging the embedding vectors of the content items of the document subset of content items, each embedding vector weighted according to the determined frequency of a corresponding content item of the document subset of content items in the current document, resulting in an averaged document embedding vector for the current document.
3 . The computer-implemented method of claim 2 , further comprising:
for each document of the identified set of documents:
determining an importance of the content item to the current document; and
averaging the embedding vectors of the content items of the document subset of content items, each embedding vector weighted according to the determined frequency of the corresponding content item in the current document and further weighted according to the importance of the content item to the current document, resulting in an averaged document embedding vector for the current document.
4 . The computer-implemented method of claim 1 , wherein:
the target content type is a text-based content type;
identifying the set of content items of the target content type from the identified set of documents comprises identifying a set of key terms as the set of content items of the identified set of documents; and
averaging the embedding vectors of the content items of the identified set of content items, thereby generating an averaged embedding vector, comprises averaging the embedding vectors of the set of key terms, thereby generating the averaged embedding vector.
5 . The computer-implemented method of claim 4 , wherein identifying the set of key terms as the set of content items of the identified set of documents comprises:
for each document of the identified set of documents:
identifying terms of a current document;
determining a term frequency/inverse document frequency (TF/IDF) score for each of the identified terms of the current document; and
identifying as key terms of the current document those identified terms whose TF/IDF score meets or exceeds a threshold score; and
aggregating the key terms of each document of the identified set of documents as the set of key terms.
6 . The computer-implemented method of claim 5 , further comprising:
for each key term of the set of key terms:
determining a number of documents of the set of identified documents in which a current key term is found; and
weighting the corresponding embedding vector of the current key term according to the determined number of documents in which the key term is found; and
wherein averaging the embedding vectors of the key terms of the set of key terms, thereby generating the averaged embedding vector, comprises averaging the weighted embedding vectors of the key terms of the set of key terms, thereby generating the averaged embedding vector.
7 . The computer-implemented method of claim 1 , further comprising:
determining, for each content item of the set of content items, a pointwise mutual information (PMI) score for the content item and the unmapped target content item; and
removing, from the set of content items, any content items having a PMI score below a threshold.