IP Library Granted Patent US 10,699,184
Granted Patent B2
US 10,699,184 · App. 15/394,289 · Granted Jun 30, 2020

Updating predictions for a deep-learning model

Inventor: Balmanohar Paluri (Mountain View, CA)
Assignee: Facebook, Inc.
G06N3/0454G06N20/00G06N99/00G06Q30/02G06Q50/01G06N7/005
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,699,184
App. No.
15/394,289
Granted
Jun 30, 2020
Kind
B2
Abstract

In one embodiment, a system retrieves a first feature vector for an image. The image is inputted into a first deep-learning model, which is a first-version model, and the first feature vector may be output from a processing layer of the first deep-learning model for the image. The first feature vector using a feature-vector conversion model to obtain a second feature vector for the image. The feature-vector conversion model is trained to convert first-version feature vectors to second-version feature vectors. The second feature vector is associated with a second deep-learning model, and the second deep-learning model is a second-version model. The second-version model is an updated version of the first-version model. A plurality of predictions for the image may be generated using the second feature vector and the second deep-learning model.

Claims (51)

1. A method comprising:

retrieving, by one or more computing devices, an image;

retrieving, by one or more of the computing devices, a first feature vector for the image, wherein:

the first feature vector is a first-version feature vector,

the first feature vector is generated by a first deep-learning model of a first version,

the first deep-learning model is trained to perform image classification with respect to a first set of concepts, and

the first feature vector comprises output from a processing layer of the first deep-learning model for the image;

retrieving, by one or more of the computing devices, a second deep-learning model requiring second-version feature vectors, wherein the second deep-learning model is an updated version of the first deep-learning model trained to perform image classification with respect to the first set of concepts and an additional set of concepts;

processing, by one or more of the computing devices, the first feature vector using a feature-vector conversion model to obtain a second feature vector for the image, wherein the second feature vector is a second-version feature vector, and wherein the feature-vector conversion model is trained to convert first-version feature vectors to second-version feature vectors; and

processing, by one or more of the computing devices, the second feature vector using the second deep-learning model, to generate a plurality of predictions for the image, wherein each of the plurality of predictions comprises a likelihood that a respective concept of the additional set of concepts is associated with the image.

2. The method of claim 1 , wherein the processing layer from which the first feature vector is outputted is a layer beneath an output layer of the first deep-learning model.

3. The method of claim 1 , further comprising:

determining that the first feature vector is the first-version feature vector, the first version being an earlier version than the second version; and

selecting the feature-vector conversion model out of a plurality of feature-vector conversion models to convert the first feature vector to the second feature vector based on the first feature vector being a first-version feature vector, wherein each of the plurality of feature-vector conversion models is trained to convert a respective particular version feature vector to the second-version feature vector.

4. The method of claim 3 , further comprising adding a feature-vector conversion layer to the second deep-learning model, wherein the feature-vector conversion layer comprises the plurality of feature-vector conversion models, and wherein the feature-vector conversion layer takes as input the first feature vector and outputs the second feature vector.

5. The method of claim 1 , wherein the second feature vector comprises an approximation of a particular second-version feature vector obtainable by processing the image using the second deep-learning model.

6. The method of claim 1 , wherein processing power used by the feature-vector conversion model to convert the first feature vector to the second feature vector for the image is less than processing power necessary to generate the second feature vector directly from the image using the second deep-learning model.

7. The method of claim 1 , wherein retrieving the first feature vector comprises accessing the image, the first feature vector being stored in association with the image.

8. The method of claim 1 , wherein the feature-vector conversion model comprises a neural network.

9. The method of claim 8 , wherein the feature-vector conversion model comprises fully-connected layers.

10. The method of claim 1 , wherein the feature-vector conversion model is trained to minimize error in converting first-version feature vectors to second-version feature vectors.

11. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

retrieve an image;

retrieve a first feature vector for the image, wherein:

the first feature vector is a first-version feature vector,

the first feature vector is generated by a first deep-learning model of a first version,

the first deep-learning model is trained to perform image classification with respect to a first set of concepts, and

the first feature vector comprises output from a processing layer of the first deep-learning model for the image;

retrieve a second deep-learning model requiring second-version feature vectors, wherein the second deep-learning model is an updated version of the first deep-learning model trained to perform image classification with respect to the first set of concepts and an additional set of concepts;

process the first feature vector using a feature-vector conversion model to obtain a second feature vector for the image, wherein the second feature vector is a second-version feature vector, and wherein the feature-vector conversion model is trained to convert first-version feature vectors to second-version feature vectors; and

process the second feature vector using the second-deep learning model, to generate a plurality of predictions for the image, wherein each of the plurality of predictions comprises a likelihood that a respective concept of the additional set of concepts is associated with the image.

12. The media of claim 11 , wherein the processing layer from which the first feature vector is outputted is a layer beneath an output layer of the first deep-learning model.

13. The media of claim 11 , wherein the software is further operable when executed to:

determine that the first feature vector is the first-version feature vector, the first version being an earlier version than the second version; and

select the feature-vector conversion model out of a plurality of feature-vector conversion models to convert the first feature vector to the second feature vector based on the first feature vector being a first-version feature vector, wherein each of the plurality of feature-vector conversion models is trained to convert a respective particular version feature vector to the second-version feature vector.

14. The media of claim 13 , wherein the software is further operable when executed to add a feature-vector conversion layer to the second deep-learning model, wherein the feature-vector conversion layer comprises the plurality of feature-vector conversion models, and wherein the feature-vector conversion layer takes as input the first feature vector and outputs the second feature vector.

15. A system comprising: one or more processors; and a memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:

retrieve an image;

retrieve a first feature vector for the image, wherein:

the first feature vector is a first-version feature vector,

the first feature vector is generated by a first deep-learning model of a first version,

the first deep-learning model is trained to perform image classification with respect to a first set of concepts, and

the first feature vector comprises output from a processing layer of the first deep-learning model for the image;

retrieve a second deep-learning model requiring second-version feature vectors, wherein the second deep-learning model is an updated version of the first deep-learning model trained to perform image classification with respect to the first set of concepts and an additional set of concepts;

process the first feature vector using a feature-vector conversion model to obtain a second feature vector for the image, wherein the second feature vector is a second-version feature vector, and wherein the feature-vector conversion model is trained to convert first-version feature vectors to second-version feature vectors; and

process the second feature vector using the second-deep learning model, to generate a plurality of predictions for the image, wherein each of the plurality of predictions comprises a likelihood that a respective concept of the additional set of concepts is associated with the image.

16. The system of claim 15 , wherein the processing layer from which the first feature vector is outputted is a layer beneath an output layer of the first deep-learning model.

17. The system of claim 15 , wherein the software is further operable when executed to:

determine that the first feature vector is the first-version feature vector, the first version being an earlier version than the second version; and

select the feature-vector conversion model out of a plurality of feature-vector conversion models to convert the first feature vector to the second feature vector based on the first feature vector being a first-version feature vector, wherein each of the plurality of feature-vector conversion models is trained to convert a respective particular version feature vector to the second-version feature vector.

18. The system of claim 17 , wherein the software is further operable when executed to add a feature-vector conversion layer to the second deep-learning model, wherein the feature-vector conversion layer comprises the plurality of feature-vector conversion models, and wherein the feature-vector conversion layer takes as input the first feature vector and outputs the second feature vector.

Assignments (2)
CHANGE OF NAME Recorded Dec 20, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058553/0802 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 21, 2017
From: PALURI, BALMANOHAR
To: FACEBOOK, INC.
Reel/Frame 041318/0907 →
Continuity (1)
Related Publication 20180189672A1 · Jul 5, 2018