IP Library Granted Patent US 10,387,464
Granted Patent B2
US 10,387,464 · App. 14/949,436 · Granted Aug 20, 2019

Predicting labels using a deep-learning model

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,387,464
App. No.
14/949,436
Granted
Aug 20, 2019
Kind
B2
Abstract

In one embodiment, a method includes receiving text query that includes n-grams. A vector representation of each n-gram is determined using a deep-learning model. A nonlinear combination of the vector representations of the n-grams is determined, and an embedding of the text query is determined based on the nonlinear combination. The embedding of the text query corresponds to a point in an embedding space, and the embedding space includes a plurality of points corresponding to a plurality of label embeddings. Each label embedding is based on a vector representation of a respective label determined using the deep-learning model. Label embeddings are identified as being relevant to the text query by applying a search algorithm to the embedding space. Points corresponding to the identified label embeddings are within a threshold distance of the point corresponding to the embedding of the text query in the embedding space.

Claims (49)

1. A method comprising, by one or more computing devices:

receiving, by one or more of the computing devices from a client system of a first user, a text query input by the first user at a user interface of the client system and comprising one or more n-grams;

determining, by one or more of the computing devices, a vector representation of each n-gram using a deep-learning model;

determining, by one or more of the computing devices, a nonlinear combination of the vector representations of the n-grams;

determining, by one or more of the computing devices, an embedding of the text query based on the nonlinear combination of the vector representations of the n-grams, wherein:

the embedding of the text query corresponds to a point in an embedding space,

the embedding space comprises a plurality of points corresponding to a plurality of label embeddings and a plurality of text embeddings, respectively,

each label embedding is based on a vector representation of a respective label determined using the deep-learning model, and

each text embedding is based on a vector representation of a respective text determined using the deep-learning model;

identifying, by one or more of the computing devices, one or more labels as relevant to the text query by applying a search algorithm to the embedding space using the deep-learning model, wherein points corresponding to one or more label embeddings of the identified labels are within a threshold distance of the point corresponding to the embedding of the text query in the embedding space, and wherein the deep-learning model has been trained to map text and labels to respective vector representations; and

sending, by one or more of the computing device to the client system of the first user, instruction for presenting a search-results interface comprising one or more suggested labels from the identified labels.

2. The method of claim 1 , wherein the deep-learning model comprises:

a first dictionary that maps text to respective vector representations; and

a second dictionary that maps labels to respective vector representations.

3. The method of claim 2 , wherein the first dictionary and the second dictionary are initialized using a random distribution.

4. The method of claim 3 , wherein the random distribution is a Gaussian distribution.

5. The method of claim 1 , wherein the deep-learning model is trained to reduce error between the vector representation of each text embedding and the vector representations of label embeddings of labels that are relevant to the respective text.

6. The method of claim 5 , wherein the deep-learning model is trained to minimize error using similarity scores determined between each text embedding and one or more label embeddings.

7. The method of claim 6 , wherein each similarity score is computed based on one or more angles between a vector representation of text associated with a text embedding and one or more vector representations of one or more labels associated with one or more label embeddings.

8. The method of claim 6 , wherein the deep-learning model is further trained using a gradient descent technique.

9. The method of claim 1 , wherein at least one of the labels comprises a hashtag.

10. The method of claim 1 , wherein at least one of the labels comprises an ideogram.

11. The method of claim 1 , further comprising sending, to the client system of the user, the one or more identified labels for display to the user.

12. The method of claim 1 , wherein the vector representations comprise d-dimensional intensity vectors.

13. The method of claim 1 , wherein the nonlinear combination of vector representations comprises a convolution.

14. The method of claim 1 , wherein the deep-learning model is a convolutional neural network.

15. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

receive, from a client system of a first user, a text query comprising input by the first user at a user interface of the client system and one or more n-grams;

determine a vector representation of each n-gram using a deep-learning model;

determine a nonlinear combination of the vector representations of the n-grams;

determine an embedding of the text query based on the nonlinear combination of the vector representations of the n-grams, wherein:

the embedding of the text query corresponds to a point in an embedding space, the embedding space comprises a plurality of points corresponding to a plurality of label embeddings and a plurality of text embeddings, respectively,

each label embedding is based on a vector representation of a respective label determined using the deep-learning model, and

each text embedding is based on a vector representation of a respective text determined using the deep-learning model;

identify one or more labels as relevant to the text query by applying a search algorithm to the embedding space using the deep-learning model, wherein points corresponding to one or more label embeddings of the identified labels are within a threshold distance of the point corresponding to the embedding of the text query in the embedding space, and wherein the deep-learning model has been trained to map text and labels to respective vector representations; and

sending, by one or more of the computing device to the client system of the first user, instruction for presenting a search-results interface comprising one or more suggested labels from the identified labels.

16. The media of claim 15 , wherein the deep-learning model comprises:

a first dictionary that maps text to respective vector representations; and

a second dictionary that maps labels to respective vector representations.

17. A system comprising: one or more processors; and a memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:

receive, from a client system of a first user, a text query comprising input by the first user at a user interface of the client system and one or more n-grams;

determine a vector representation of each n-gram using a deep-learning model;

determine a nonlinear combination of the vector representations of the n-grams;

determine an embedding of the text query based on the nonlinear combination of the vector representations of the n-grams, wherein:

the embedding of the text query corresponds to a point in an embedding space, the embedding space comprises a plurality of points corresponding to a plurality of label embeddings and a plurality of text embeddings, respectively,

each label embedding is based on a vector representation of a respective label determined using the deep-learning model, and

each text embedding is based on a vector representation of a respective text determined using the deep-learning model;

identify one or more labels as relevant to the text query by applying a search algorithm to the embedding space using the deep-learning model, wherein points corresponding to one or more label embeddings of the identified labels are within a threshold distance of the point corresponding to the embedding of the text query in the embedding space, and wherein the deep-learning model has been trained to map text and labels to respective vector representations; and

sending, by one or more of the computing device to the client system of the first user, instruction for presenting a search-results interface comprising one or more suggested labels from the identified labels.

Assignments (2)
CHANGE OF NAME Recorded Dec 20, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058553/0802 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 28, 2016
From: WESTON, JASON E.; ADAMS, KEITH; CHOPRA, SUMIT
To: FACEBOOK, INC.
Reel/Frame 038111/0131 →