IP Library Granted Patent US 10,558,759
Granted Patent B1
US 10,558,759 · App. 15/862,057 · Granted Feb 11, 2020

Consumer insights analysis using word embeddings

Inventors: Jonathan Michael Arfa (New York, NY); Nikhil Girish Nawathe (New York, NY); Bryan Kauder (New York, NY); Shriram Subramanian (New York, NY); Alyx Catherine Stevens (New York, NY)
Assignee: Facebook, Inc.
G06F17/2785G06F17/278G06N20/00H04L51/32
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,558,759
App. No.
15/862,057
Granted
Feb 11, 2020
Kind
B1
Abstract

In one embodiment, a method includes receiving a request to generate k words that each approximates a representation of a relationship between two concepts, where the request includes two input n-grams that each represent one of the two concepts, accessing a table of word vector relationships, where the table includes a plurality of unique n-grams and their corresponding word vectors, looking up word vectors corresponding to each of the two input n-grams using the table, calculating an average vector of the word vectors corresponding to the two input n-grams, selecting, using the table and based on a similarity metric, k word vectors closest to the average vector in the embedding space, identifying, for each of the selected word vectors, a corresponding n-gram by looking up the selected word vector in the table, and sending a response message, the response message comprising the identified n-grams.

Claims (49)

1. A method comprising:

by a first computing device in an online social network, receiving, from a second computing device, a request to generate k words that each approximates a representation of a relationship between two concepts, wherein the request comprises two input n-grams that each represents one of the two concepts, and wherein the request comprises one or more conditions characterizing an audience;

by the first computing device, identifying users of the online social network who satisfy the one or more conditions;

by the first computing device, constructing a corpus of text by collecting text content from content objects created by the identified users;

by the first computing device, constructing a table of word vector relationships by training a word embedding mode using the constructed corpus of text as training data, wherein the table of word vector relationships comprises a plurality of unique n-grams and their corresponding word vectors, wherein each of the word vectors represents a semantic context of a corresponding n-gram as a point in a d-dimensional embedding space;

by the first computing device, looking up, using the table, word vectors corresponding to each of the two input n-grams;

by the first computing device, calculating an average vector by taking a weighted average of the word vectors corresponding to the two input n-grams;

by the first computing device, selecting, using the table and based on a similarity metric, k word vectors closest to the average vector in the embedding space;

by the first computing device, identifying, for each of the selected word vectors, a corresponding n-gram by looking up the selected word vector in the table; and

by the first computing device, sending, to the second computing device, a response message, the response message comprising the identified n-grams.

2. The method of claim 1 , wherein the plurality of unique n-grams in the table are selected from the constructed corpus of text.

3. The method of claim 1 , wherein the word embedding model is a word2vec model.

4. The method of claim 1 , wherein the looking up a word vector corresponding to each of the two input n-grams comprises looking up the n-gram in the table.

5. The method of claim 1 , wherein a weight assigned to each of the two word vectors corresponding to the two input n-grams is an Inverse Document Frequency (IDF) score for the corresponding n-gram.

6. The method of claim 5 , wherein the IDF score for an n-gram is a measure reflecting specificity of the n-gram, wherein the IDF score is based on a number of documents containing the n-gram in a corpus of text.

7. The method of claim 1 , wherein the similarity metric is a cosine similarity.

8. The method of claim 1 , wherein the similarity metric is a Euclidean distance.

9. The method of claim 1 , wherein the similarity metric is a Jaccard similarity coefficient.

10. The method of claim 1 , wherein each of the two concepts respectively represents a projected target audience.

11. The method of claim 1 , wherein the constructing the table of word vector relationships comprises:

identifying a list of unique n-grams appearing in the corpus of text;

training a word embedding model using the corpus of text, wherein the word embedding model generates word vectors corresponding to the unique n-grams; and

generating, using the word embedding model, the table based on the list of unique n-grams and their corresponding word vectors.

12. The method of claim 1 , wherein the content objects were created within a pre-determined period of time.

13. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

receive, from a second computing device, a request to generate k words that each approximates a representation of a relationship between two concepts, wherein the request comprises two input n-grams that each represents one of the two concepts, and wherein the request comprises one or more conditions characterizing an audience;

by the first computing device, identifying users of the online social network who satisfy the one or more conditions;

by the first computing device, constructing a corpus of text by collecting text content from content objects created by the identified users;

construct a table of word vector relationships by training a word embedding mode using the constructed corpus of text as training data, wherein the table of word vector relationships comprises a plurality of unique n-grams and their corresponding word vectors, wherein each of the word vectors represents a semantic context of a corresponding n-gram as a point in a d-dimensional embedding space;

look up, using the table, word vectors corresponding to each of the two input n-grams;

calculate an average vector by taking a weighted average of the word vectors corresponding to the two input n-grams;

select, using the table and based on a similarity metric, k word vectors closest to the average vector in the embedding space;

identify, for each of the selected word vectors, a corresponding n-gram by looking up the selected word vector in the table; and

send, to the second computing device, a response message, the response message comprising the identified n-grams.

14. The media of claim 13 , wherein the plurality of unique n-grams in the table are selected from the constructed corpus of text.

15. The media of claim 13 , wherein the word embedding model is a word2vec model.

16. The media of claim 13 , wherein the looking up a word vector corresponding to each of the two input n-grams comprises looking up the n-gram in the table.

17. A system comprising:

one or more processors; and

one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:

receive, from a second computing device, a request to generate k words that each approximates a representation of a relationship between two concepts, wherein the request comprises two input n-grams that each represents one of the two concepts, and wherein the request comprises one or more conditions characterizing an audience;

by the first computing device, identifying users of the online social network who satisfy the one or more conditions;

by the first computing device, constructing a corpus of text by collecting text content from content objects created by the identified users;

construct a table of word vector relationships by training a word embedding mode using the constructed corpus of text as training data, wherein the table of word vector relationships comprises a plurality of unique n-grams and their corresponding word vectors, wherein each of the word vectors represents a semantic context of a corresponding n-gram as a point in a d-dimensional embedding space;

look up, using the table, word vectors corresponding to each of the two input n-grams;

calculate an average vector by taking a weighted average of the word vectors corresponding to the two input n-grams;

select, using the table and based on a similarity metric, k word vectors closest to the average vector in the embedding space;

identify, for each of the selected word vectors, a corresponding n-gram by looking up the selected word vector in the table; and

send, to the second computing device, a response message, the response message comprising the identified n-grams.

Assignments (2)
CHANGE OF NAME Recorded Dec 20, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058553/0802 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 6, 2018
From: ARFA, JONATHAN MICHAEL; NAWATHE, NIKHIL GIRISH; KAUDER, BRYAN; STEVENS, ALYX CATHERINE; SUBRAMANIAN, SHRIRAM
To: FACEBOOK, INC.
Reel/Frame 045464/0661 →
Cited By (5)
US 12,299,036 US 12,339,904 US 12,379,902 US 12,386,874 US 12,586,566