IP Library › Granted Patent US 11,768,874
Granted Patent B2
US 11,768,874 · App. 16/225,888 · Granted Sep 26, 2023

Compact entity identifier embeddings

Inventors: Yiming Ma (Menlo Park, CA); Xuhong Zhang (Sunnyvale, CA); Wei Lu (Santa Clara, CA); Mingzhou Zhou (Sunnyvale, CA)
Assignee: Microsoft Technology Licensing, LLC
G06F16/9017G06F16/906G06F16/9014G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,768,874
App. No.
16/225,888
Granted
Sep 26, 2023
Kind
B2
Abstract

The disclosed embodiments provide a system for processing data. During operation, the system applies a first set of hash functions to a first entity identifier (ID) for a first entity to generate a first set of hash values. Next, the system produces a first set of intermediate vectors from the first set of hash values and a first set of lookup tables by matching each hash value in the first set of hash values to an entry in a corresponding lookup table in the first set of lookup tables. The system then performs an element-wise aggregation of the first set of intermediate vectors to produce a first embedding. Finally, the system outputs the first embedding for use by a machine learning model.

Claims (73)

1. A method, comprising:

applying, by one or more computer systems, a first set of hash functions to a first entity identifier (ID) for a first entity to generate a first set of hash values;

using the first set of hash values as indexes, retrieving, from a first set of lookup tables, a first set of intermediate vectors that represent the first entity ID, wherein the first set of intermediate vectors has a dimensionality comprising a common number of dimensions associated with the first entity and a dimension comprises a numeric value;

aggregating corresponding dimensions of the first set of intermediate vectors using an element-wise aggregation to produce a first embedding that represents the first entity ID and has the same dimensionality as the first set of intermediate vectors;

applying a machine learning model to an embedding pair comprising the first embedding and a second embedding that represents a second entity ID, wherein the second embedding is produced by element-wise aggregation of a second set of intermediate vectors retrieved from a second set of lookup tables using a second set of hash values as indexes and has the same dimensionality as the first embedding;

based on the embedding pair, generating, by the machine learning model, output that represents a likelihood value, wherein the likelihood value is associated with a positive label as an indication of a positive interaction between the first entity and a second entity associated with the second entity ID; and

training the machine learning model to predict an outcome associated with the first entity and the second entity based on the first embedding and an additional input.

2. The method of claim 1 , further comprising:

updating, based on the output, parameters in the machine learning model for generating the first embedding from the first entity ID.

3. The method of claim 2 , further comprising:

updating, based on the output, additional parameters in the machine learning model for generating the second embedding from the second entity ID for the second entity.

4. The method of claim 1 , further comprising:

updating, based on the output, parameters for calculating subsequent output from embedding pairs.

5. The method of claim 1 , further comprising:

applying a second set of hash functions to the second entity ID for the second entity to produce the second set of hash values;

producing the second set of intermediate vectors from the second set of hash values and the second set of lookup tables associated with the second set of hash values; and

combining the second set of intermediate vectors into the second embedding.

6. The method of claim 1 , wherein the second embedding comprises features associated with the second entity.

7. The method of claim 1 , wherein the first entity comprises at least one of:

a member;

a job;

a company; or

a content item.

8. The method of claim 1 , wherein the element-wise aggregation comprises an element-wise maximum of a set of intermediate vectors.

9. The method of claim 1 , wherein:

the machine learning model is trained on labeled entity pairs;

a labeled entity pair comprises a pair of entity identifiers and either a positive label or a negative label;

a positive label represents a positive outcome between the entities associated with the pair of entity identifiers; and

a negative label represents a negative outcome between the entities associated with the pair of entity identifiers.

10. A system comprising:

one or more processors; and

memory storing instructions that, when executed by the one or more processors, cause the system to:

using a first set of hash values as indexes, retrieve, from a first set of lookup tables, a first set of intermediate vectors that represent a first entity identifier (ID), wherein the first set of intermediate vectors has a dimensionality comprising a common number of dimensions associated with the first entity and a dimension comprises a numeric value;

aggregate corresponding dimensions of the first set of intermediate vectors using an element-wise aggregation to produce a first embedding that represents the first entity ID and has the same dimensionality as the first set of intermediate vectors;

apply a machine learning model to an embedding pair comprising the first embedding and a second embedding that represents a second entity ID, wherein the second embedding is produced by element-wise aggregation of a second set of intermediate vectors retrieved from a second set of lookup tables using a second set of hash values as indexes and has the same dimensionality as the first embedding;

based on the embedding pair, generate, by the machine learning model, output that represents a likelihood value, wherein the likelihood value is associated with a positive label as an indication of a positive interaction between the first entity and a second entity associated with the second entity ID; and

train the machine learning model to predict an outcome associated with the first entity and the second entity based on the first embedding and an additional input.

11. The system of claim 10 , wherein the instructions, when executed by the one or more processors, cause the system to:

update, based on the outcome output, parameters in the machine learning model for generating the first embedding from the first entity ID and generating the second embedding.

12. The system of claim 10 , wherein the instructions, when executed by the one or more processors, cause the system to:

update, based on the output, parameters for calculating subsequent output from embedding pairs.

13. The system of claim 10 , wherein the instructions, when executed by the one or more processors, cause the system to:

apply a second set of hash functions to the second entity ID for the second entity to produce the second set of hash values;

produce the second set of intermediate vectors from the second set of hash values and the second set of lookup tables associated with the second set of hash values; and

combine the second set of intermediate vectors into the second embedding.

14. The system of claim 10 , wherein the second embedding comprises features associated with the second entity.

15. The system of claim 10 , wherein the element-wise aggregation comprises an element-wise maximum of a set of intermediate vectors.

16. The system of claim 10 , wherein:

the machine learning model is trained on labeled entity pairs;

a labeled entity pair comprises a pair of entity identifiers and either a positive label or a negative label;

a positive label represents a positive outcome between the entities associated with the pair of entity identifiers; and

a negative label represents a negative outcome between the entities associated with the pair of entity identifiers.

17. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform operations comprising:

applying, by one or more computer systems, a first set of hash functions to a first entity identifier (ID) for a first entity to generate a first set of hash values;

using the first set of hash values as indexes, retrieving, from a first set of lookup tables, a first set of intermediate vectors that represent the first entity ID, wherein the first set of intermediate vectors has a dimensionality comprising a common number of dimensions associated with the first entity and a dimension comprises a numeric value;

aggregating corresponding dimensions of the first set of intermediate vectors using an element-wise aggregation to produce a first embedding that represents the first entity ID and has the same dimensionality as the first set of intermediate vectors;

applying a machine learning model to an embedding pair comprising the first embedding and a second embedding that represents a second entity ID, wherein the second embedding is produced by element-wise aggregation of a second set of intermediate vectors retrieved from a second set of lookup tables using a second set of hash values as indexes and has the same dimensionality as the first embedding;

based on the embedding pair, generating, by the machine learning model, output that represents a likelihood value, wherein the likelihood value is associated with a positive label as an indication of a positive interaction between the first entity and a second entity associated with the second entity ID; and

training the machine learning model to predict an outcome associated with the first entity and the second entity based on the first embedding and an additional input.

18. The non-transitory computer-readable storage medium of claim 17 , wherein:

the machine learning model is trained on labeled entity pairs to generate output that represents a likelihood of the first entity having a positive interaction with a second entity associated with a second entity ID;

a labeled entity pair comprises a pair of entity identifiers and either a positive label or a negative label;

a positive label represents a positive outcome between the entities associated with the pair of entity identifiers; and

a negative label represents a negative outcome between the entities associated with the pair of entity identifiers.

19. The non-transitory computer-readable storage medium of claim 17 , wherein the instructions, when executed by a computer cause the computer to perform operations further comprising at least one of:

updating, based on the output, parameters in the machine learning model for generating the first embedding from the first entity ID; or

updating, based on the output, additional parameters in the machine learning model for generating the second embedding from the second entity ID for the second entity; or

updating, based on the output, parameters for calculating subsequent output from embedding pairs; or

(i) applying a second set of hash functions to the second entity ID for the second entity to produce the second set of hash values, (ii) producing the second set of intermediate vectors from the second set of hash values and the second set of lookup tables associated with the second set of hash values, and (iii) combining the second set of intermediate vectors into the second embedding.

20. The non-transitory computer-readable storage medium of claim 17 , wherein at least one of:

the second embedding comprises features associated with the second entity; or

the first entity comprises at least one of a member, a job, a company, or a content item; or

the element-wise aggregation comprises an element-wise maximum of a set of intermediate vectors.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 16, 2019
From: MA, YIMING; ZHANG, XUHONG; LU, WEI; ZHOU, MINGZHOU
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 048037/0900 →
Continuity (1)
Related Publication 20200201908A1 · Jun 25, 2020