IP Library Granted Patent US 10,043,254
Granted Patent B2
US 10,043,254 · App. 15/098,950 · Granted Aug 7, 2018

Optimal image transformation based on professionalism score of subject

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,043,254
App. No.
15/098,950
Granted
Aug 7, 2018
Kind
B2
Abstract

In an example embodiment, an image transformation is automatically performed on a digital image to improve perceived professionalism of a subject of the image. A machine learning algorithm is utilized to generate a professionalism score for the digital image, the utilizing a machine learning algorithm comprising: a training mode where a plurality of sample images with labeled professionalism scores are used to train a classification function in a model that produces as professionalism score as output; an analysis mode where the model is used to generate a professionalism score for the digital image. Then the professionalism score is used as an input to a continuous variable optimization algorithm to determine an optimum version of the digital image from a plurality of possible versions of the digital image on which one or more image transformations have been performed, using the classification function.

Claims (47)

1. A computerized method for automatically performing an image transformation on a digital image to improve perceived professionalism of a subject of the image, the method comprising:

utilizing a machine learning algorithm to generate a professionalism score for the digital image, wherein the professionalism score indicates a perceived professionalism of a human depicted in the digital image, the utilizing a machine learning algorithm comprising:

a training mode where a plurality of sample images with labeled professionalism scores are used to train a classification function in a model that produces as professionalism score as output;

an analysis mode where the model is used to generate a professionalism score for the digital image; and

using the professionalism score for the digital image as an input to a continuous variable optimization algorithm to determine an optimum version of the digital image from a plurality of possible versions of the digital image on which one or more image transformations have been performed, using the classification function, wherein the continuation variable optimization algorithm uses a deep convolutional neural network (DCNN) by passing the digital image to a convolutional layer, generating output, passing the output from the convolutional layer to a nonlinearity layer, generating output, passing the output from the nonlinearity layer to a pooling layer, generating output, and passing output from the nonlinearity layer to a classification layer.

2. The method of claim 1 , wherein the classification function contains one or more weights, and wherein the training of the classification function incudes modifying the one or more weights.

3. The method of claim 1 , wherein the one or more image transformations includes image rotation.

4. The method of claim 1 , wherein the one or more image transformations includes lighting compensation.

5. The method of claim 1 , wherein the one or more image transformations includes edge enhancing filtering.

6. The method of claim 1 , wherein the one or more image transformations includes edge enhancing filtering.

7. The method of claim 1 , wherein the one or more image transformations includes spatially varying blur filtering.

8. The method of claim 1 , wherein the one or more image transformations includes at least two image transformations selected from the group consisting of:

image rotation;

lighting compensation;

edge enhancing filtering;

edge enhancing filtering; and

spatially varying blur filtering.

9. The method of claim 1 , wherein the continuous variable optimization algorithm is a derivative-free optimization algorithm.

10. The method of claim 1 , wherein the continuous variable optimization algorithm is a constrained optimization algorithm.

11. A non-transitory machine-readable storage medium comprising instructions, which when implemented by one or more machines, cause the one or more machines to perform operations for automatically performing an image transformation on a digital image to improve perceived professionalism of a subject of the image, the operations comprising:

utilizing a machine learning algorithm to generate a professionalism score for the digital image, wherein the professionalism score indicates a perceived professionalism of a human depicted in the digital image, the utilizing a machine learning algorithm comprising:

a training mode where a plurality of sample images with labeled professionalism scores are used to train a classification function in a model that produces as professionalism score as output;

an analysis mode where the model is used to generate a professionalism score for the digital image; and

using the professionalism score for the digital image as an input to a continuous variable optimization algorithm to determine an optimum version of the digital image from a plurality of possible versions of the digital image on which one or more image transformations have been performed, using the classification function, wherein the continuation variable optimization algorithm uses a deep convolutional neural network (DCNN) by passing the digital image to a convolutional layer, generating output, passing the output from the convolutional layer to a nonlinearity layer, generating output, passing the output from the nonlinearity layer to a pooling layer, generating output, and passing output from the nonlinearity layer to a classification layer.

12. The non-transitory machine-readable storage medium of claim 11 , wherein the classification function contains one or more weights, and wherein the training of the classification function incudes modifying the one or more weights.

13. The non-transitory machine-readable storage medium of claim 11 , wherein the one or more image transformations includes image rotation.

14. The non-transitory machine-readable storage medium of claim 11 , wherein the one or more image transformations includes lighting compensation.

15. The non-transitory machine-readable storage medium of claim 11 , wherein the one or more image transformations includes edge enhancing filtering.

16. The non-transitory machine-readable storage medium of claim 11 , wherein the one or more image transformations includes edge enhancing filtering.

17. The non-transitory machine-readable storage medium of claim 11 , wherein the one or more image transformations includes spatially varying blur filtering.

18. The non-transitory machine-readable storage medium of claim 11 , wherein the one or more image transformations includes at least two image transformations selected from the group consisting of:

image rotation;

lighting compensation;

edge enhancing filtering;

edge enhancing filtering; and

spatially varying blur filtering.

19. The non-transitory machine-readable storage medium of claim 11 , wherein the continuous variable optimization algorithm is a derivative-free optimization algorithm.

20. The non-transitory machine-readable storage medium of claim 11 , wherein the continuous variable optimization algorithm is a constrained optimization algorithm comprising:

training a DCNN by:

inputting a plurality of sample images to the DCNN, each of the sample images having been labeled with a professionalism score, the inputting including, for each sample image:

passing the image to a convolutional layer of the DCNN, the convolutional layer comprising one or more filters having dynamically adjustable weights, the one or more filters configured to filter the image to produce an output volume for the corresponding image, the output volume comprising a different feature map for each of the one or more filters;

passing the output volume from the convolutional layer through a nonlinearity layer, the nonlinearity layer applying a nonlinearity function to the output volume from the convolutional layer;

passing the output volume from the nonlinearity layer through a pooling layer, the pooling layer lowering spatial dimensions of the output volume from the nonlinearity layer;

passing the output volume from the pooling layer through a classification layer, the classification layer comprising a specialized convolutional layer having a filter designed to output a professionalism score for the image based on the output volume from the pooling layer; and

passing the image through a loss layer, the loss layer applying a loss function to the image, resulting an in indication of a level of error in the professionalism score for the image from the classification layer in comparison to the professionalism score from the label of the image;

determining whether a combination of the levels of error for the plurality of sample images transgresses a preset threshold; and

in response to a determination that the combination of the levels of error transgresses a preset threshold, updating weights of the one or more filters in the convolutional layers of the DCNN to reduce the combination of the levels of error and repeating the training of the DCNN using a different plurality of sample images and the updated weights.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 1, 2017
From: LINKEDIN CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 044746/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 14, 2016
From: MERHAV, URI; SHACHAM, DAN
To: LINKEDIN CORPORATION
Reel/Frame 038285/0189 →