IP Library Granted Patent US 11,030,539
Granted Patent B1
US 11,030,539 · App. 15/862,066 · Granted Jun 8, 2021

Consumer insights analysis using word embeddings

Inventors: Jonathan Michael Arfa (New York, NY); Nikhil Girish Nawathe (New York, NY); Bryan Kauder (New York, NY); Shriram Subramanian (New York, NY)
Assignee: Facebook, Inc.
G06N5/04G06F40/295G06F40/30G06N20/00G06Q30/0201G06Q50/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,030,539
App. No.
15/862,066
Granted
Jun 8, 2021
Kind
B1
Abstract

In one embodiment, a method includes receiving a request to identify a word representing a target concept that is in a first relationship with a particular concept such that the first relationship is analogous to a second relationship in which a first reference concept is with a second reference concept, accessing a table of word vector relationships, looking up a particular word vector, a first reference word vector, and a second reference word vector, determining an imaginary vector such that a first vector from the first reference word vector to the second reference word vector is equal to a second vector from the particular word vector to the imaginary vector, selecting a target word vector closest to the imaginary vector, identifying a target n-gram corresponding to the target word vector, and sending a response message comprising the target n-gram.

Claims (55)

1. A method comprising:

by a computing device in an online social network, receiving a request to identify a word representing a target concept that is in a first relationship with a particular concept based on that a first reference concept is in a second relationship with a second reference concept, the first relationship being analogous to the second relationship, wherein the request comprises a particular n-gram representing the particular concept, a first reference n-gram representing the first reference concept, and a second reference n-gram representing the second reference concept;

by the computing device, accessing a table of word vector relationships, wherein the table comprises a plurality of unique n-grams and their corresponding word vectors, wherein each of the word vectors represents a semantic context of a corresponding n-gram as a point in a d-dimensional embedding space;

by the computing device, looking up, using the table, a particular word vector corresponding to the particular n-gram, a first reference word vector corresponding to the first reference n-gram, and a second reference word vector corresponding to the second reference n-gram;

by the computing device, determining an imaginary vector such that a first directional vector from a point represented by the first reference word vector to a point represented by the second reference word vector is equal to a second directional vector from a point represented by the particular word vector to a point represented by the imaginary vector, wherein the first directional vector and the second directional vector are equal if the vectors have the same magnitude and the same direction;

by the computing device, selecting, using the table and based on a similarity metric, a target word vector closest to the imaginary vector;

by the computing device, identifying a target n-gram corresponding to the target word vector by looking up the target word vector in the table; and

by the computing device, sending a response message comprising the target n-gram.

2. The method of claim 1 , wherein the plurality of unique n-grams in the table are selected from a corpus of text, wherein the corpus of text was collected from a plurality of user-created content objects in the online social network.

3. The method of claim 2 , wherein the table represents results of a word embedding model trained using the corpus of text as training data.

4. The method of claim 3 , wherein the word embedding model is a word2vec model.

5. The method of claim 1 , wherein the looking up the particular word vector, the first reference word vector, and the second reference word vector comprises looking up the particular n-gram, the first reference n-gram, and the second reference n-gram in the table.

6. The method of claim 1 , wherein the determining the imaginary vector comprises:

computing a temporary vector by subtracting the first reference vector from the particular word vector; and

computing the imaginary vector by adding the second reference vector to the temporary vector.

7. The method of claim 1 , wherein the selecting the target word vector comprises:

computing, for each word vector in the table, a similarity metric to the imaginary vector; and

selecting a word vector that has a highest similarity metric to the imaginary vector.

8. The method of claim 7 , wherein the similarity metric is a cosine similarity, a Euclidean distance, or a Jaccard similarity coefficient.

9. The method of claim 1 , wherein the request comprises one or more conditions characterizing an audience.

10. The method of claim 9 , further comprising:

constructing the corpus of text by collecting text content from content objects created by a group of users of the online social network, wherein the group satisfies the one or more conditions;

identifying a list of unique n-grams appearing in the corpus of text;

training a word embedding model using the corpus of text, wherein the word embedding model generates word vectors corresponding to the unique n-grams; and

generating, using the word embedding model, the table based on the list of unique n-grams and their corresponding word vectors.

11. The method of claim 10 , wherein the content objects were created within a pre-determined period of time.

12. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

receive a request to identify a word representing a target concept that is in a first relationship with a particular concept based on that a first reference concept is in a second relationship with a second reference concept, the first relationship being analogous to the second relationship, wherein the request comprises a particular n-gram representing the particular concept, a first reference n-gram representing the first reference concept, and a second reference n-gram representing the second reference concept;

access a table of word vector relationships, wherein the table comprises a plurality of unique n-grams and their corresponding word vectors, wherein each of the word vectors represents a semantic context of a corresponding n-gram as a point in a d-dimensional embedding space;

look up, using the table, a particular word vector corresponding to the particular n-gram, a first reference word vector corresponding to the first reference n-gram, and a second reference word vector corresponding to the second reference n-gram;

determine an imaginary vector such that a first directional vector from a point represented by the first reference word vector to a point represented by the second reference word vector is equal to a second directional vector from a point represented by the particular word vector to a point represented by the imaginary vector, wherein the first directional vector and the second directional vector are equal if the vectors have the same magnitude and the same direction;

select, using the table and based on a similarity metric, a target word vector closest to the imaginary vector;

identify a target n-gram corresponding to the target word vector by looking up the target word vector in the table; and

send a response message comprising the target n-gram.

13. The media of claim 12 , wherein the plurality of unique n-grams in the table are selected from a corpus of text, wherein the corpus of text was collected from a plurality of user-created content objects in the online social network.

14. The media of claim 13 , wherein the table represents results of a word embedding model trained using the corpus of text as training data.

15. The media of claim 14 , wherein the word embedding model is a word2vec model.

16. The media of claim 12 , wherein the looking up the particular word vector, the first reference word vector, and the second reference word vector comprises looking up the particular n-gram, the first reference n-gram, and the second reference n-gram in the table.

17. The media of claim 12 , wherein the determining the imaginary vector comprises:

computing a temporary vector by subtracting the first reference vector from the particular word vector; and

computing the imaginary vector by adding the second reference vector to the temporary vector.

18. The media of claim 12 , wherein the selecting the target word vector comprises:

computing, for each word vector in the table, a similarity metric to the imaginary vector; and

selecting a word vector that has a highest similarity metric to the imaginary vector.

19. The media of claim 18 , wherein the similarity metric is a cosine similarity, a Euclidean distance, or a Jaccard similarity coefficient.

20. A system comprising:

one or more processors; and

one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:

receive a request to identify a word representing a target concept that is in a first relationship with a particular concept based on that a first reference concept is in a second relationship with a second reference concept, the first relationship being analogous to the second relationship, wherein the request comprises a particular n-gram representing the particular concept, a first reference n-gram representing the first reference concept, and a second reference n-gram representing the second reference concept;

access a table of word vector relationships, wherein the table comprises a plurality of unique n-grams and their corresponding word vectors, wherein each of the word vectors represents a semantic context of a corresponding n-gram as a point in a d-dimensional embedding space;

look up, using the table, a particular word vector corresponding to the particular n-gram, a first reference word vector corresponding to the first reference n-gram, and a second reference word vector corresponding to the second reference n-gram;

determine an imaginary vector such that a first directional vector from a point represented by the first reference word vector to a point represented by the second reference word vector is equal to a second directional vector from a point represented by the particular word vector to a point represented by the imaginary vector, wherein the first directional vector and the second directional vector are equal if the vectors have the same magnitude and the same direction;

select, using the table and based on a similarity metric, a target word vector closest to the imaginary vector;

identify a target n-gram corresponding to the target word vector by looking up the target word vector in the table; and

send a response message comprising the target n-gram.

Assignments (2)
CHANGE OF NAME Recorded Dec 20, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058553/0802 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 6, 2018
From: ARFA, JONATHAN MICHAEL; NAWATHE, NIKHIL GIRISH; KAUDER, BRYAN; SUBRAMANIAN, SHRIRAM
To: FACEBOOK, INC.
Reel/Frame 045465/0290 →