IP Library › Granted Patent US 12,160,409
Granted Patent B2
US 12,160,409 · App. 17/676,157 · Granted Dec 3, 2024

System and method for anonymization of a face in an image

Inventors: Eliran Kuta (Tel Aviv, IL); Sella Blondheim (Tel Aviv, IL); Gil Perry (Tel Aviv, IL); Yoav Hacohen (Jerusalem, IL)
Assignee: DE-IDENTIFICATION LTD.
H04L63/0421G06N3/08G06V10/764G06V10/774G06V10/82G06V40/16G06V40/162G06V40/171G06V40/172
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,160,409
App. No.
17/676,157
Filed
Feb 20, 2022
Granted
Dec 3, 2024
Kind
B2
Art Unit
2662
USPC
382/118
Abstract

A system and method of anonymization of a face in a set of images by at least one processor, the method including: receiving a first set of images; extracting from the first set of images a first face, depicting a person and having a first set of attributes; and performing a perturbation of the first face to produce a second face having a second set of attributes, wherein the second set of attributes is adapted to be visually perceived by a viewing entity as being substantially equivalent to the first set of attributes, and wherein the second face is adapted to be visually perceived by a viewing entity as not pertaining to the depicted person.

Claims (54)

1. A method of anonymizing a face in a set of images by at least one processor, the method comprising:

training a machine-learning (ML) based face generator model comprising a face encoder and a face decoder to reconstruct an input image of a face;

reducing a dimension of the face encoder module to generate a reduced dimension face generator model and retraining the reduced dimension face generator model so that:

an image of a face generated by the reduced dimension face generator model from the input image is visually perceived by a viewing entity as not pertaining to the same person as in the input image, and

attributes of the face in the generated image is perceived by an attribute classifier as substantially equivalent to the attributes of the input image;

repeating reducing and retraining until a predefined encoder dimension limit is reached;

extracting, from a first set of images, a new face depicting a new person and having a first set of attributes; and

using the reduced dimension face generator model to produce an anonymized face from the new face, the anonymized face having a second set of attributes,

wherein the second set of attributes is adapted to be perceived, by the attribute classifier, as substantially equivalent to the first set of attributes, and wherein the anonymized face is adapted to be visually perceived by the viewing entity as not pertaining to the new person.

2. The method of claim 1 , wherein the face encoder module comprises a neural network (NN), wherein reducing the dimension of the face encoder module comprises at least on of: omitting at least one node in at least one layer of the NN, omitting at least one layer of the NN or omitting at least one link in the NN.

3. The method of claim 1 , wherein the face encoder module comprises a neural network (NN) and wherein the minimal encoder dimension limit is a predefined number of NN nodes.

4. The method of claim 1 , further comprising blending a background of the first set of images with the anonymized face to produce a second set of images.

5. The method of claim 1 , wherein the viewing entity is selected from a list consisting of: a perceptual similarity metric module, a human vision similarity predictor; a face recognition classifier; and feedback from a human viewer.

6. The method of claim 1 , wherein the attributes of the first set of attributes and second set of attributes are selected from a list consisting of: facial attributes, positioning attributes and accessory attributes.

7. The method of claim 6 , wherein the facial attributes are selected from a list consisting of: an age, a gender, an ethnicity, an emotion, an expression, a complexion and an eye color, wherein the positioning attributes are selected from a list consisting of: a position of a face, an orientation of a face, a pose of a face, and an elevation of a face, and wherein the accessory attributes are selected from a list consisting of: existence of spectacles on a face, existence of jewelry on the face and existence of hair dressing accessories on the face.

8. The method of claim 1 , wherein training the ML-based face generator module comprises:

receiving at least one labeled image of a face, pertaining to the labeled training set of images, as a first input;

receiving at least one output indication of an attribute classifier as a second input; and

receiving at least one output indication of a viewing entity as a third input.

9. The method of claim 1 , wherein retraining the reduced dimension face generator model comprises using the already trained face decoder and training the reduced dimension face encoder using a cost function that decreases as a level of similarity between the attributes of the face in the generated image and the attributes of the input image decreases and increases as the level of similarity in identity between the face in the generated image and the face in the input image decreases.

10. The method of claim 1 , wherein producing the anonymized face comprises:

using the face encoder, to produce, from the new face, a faceprint vector having a reduced dimension in relation to the new face; and

using the face decoder to generate the anonymized face from the faceprint vector, wherein the face decoder comprises a generative neural network (GNN) module.

11. A method of anonymizing a face in a set of images by at least one processor, the method comprising:

training a machine-learning (ML) based face generator model comprising a face encoder and a generative neural network (GNN) module to reconstruct an input image of a face;

reducing a dimension of the face encoder module to generate a reduced dimension face generator model and retraining the reduced dimension face generator model so that:

an image of a face generated by the reduced dimension face generator model from the input image is visually perceived by a viewing entity as not pertaining to the same person as in the input image, and

attributes of the face in the generated image is perceived by an attribute classifier as substantially equivalent to the attributes of the input image;

repeating reducing and retraining until a predefined encoder dimension limit is reached;

extracting, from a first set of images, a new face depicting a new person and having a first set of attributes; and

using the reduced dimension face generator model to produce an anonymized face from the new face, the anonymized face having a second set of attributes,

wherein the second set of attributes is adapted to be perceived, by the attribute classifier, as substantially equivalent to the first set of attributes, and wherein the anonymized face is adapted to be visually perceived by the viewing entity as not pertaining to the new person.

12. A system of anonymizing a face in a set of images, comprising:

a memory; and

a processor configured to:

train a machine-learning (ML) based face generator model comprising a face encoder and a face decoder to reconstruct an input image of a face;

reduce a dimension of the face encoder module to generate a reduced dimension face generator model and retraining the reduced dimension face generator model so that:

an image of a face generated by the reduced dimension face generator model from the input image is visually perceived by a viewing entity as not pertaining to the same person as in the input image, and

attributes of the face in the generated image is perceived by an attribute classifier as substantially equivalent to the attributes of the input image;

repeat reducing and retraining until a predefined encoder dimension limit is reached;

extract, from a first set of images, a new face depicting a new person and having a first set of attributes; and

use the reduced dimension face generator model to produce an anonymized face from the new face, the anonymized face having a second set of attributes,

wherein the second set of attributes is adapted to be perceived, by the attribute classifier, as substantially equivalent to the first set of attributes, and wherein the anonymized face is adapted to be visually perceived by the viewing entity as not pertaining to the new person.

13. The system of claim 12 , wherein the face encoder module comprises a neural network (NN), wherein the processor is configured to reduce the dimension of the face encoder module by performing at least one of: omitting at least one node in at least one layer of the NN, omitting at least one layer of the NN or omitting at least one link in the NN.

14. The system of claim 12 , wherein the processor is configured to blend a background of the first set of images with the anonymized face to produce a second set of images.

15. The system of claim 12 , wherein the attributes of the first set of attributes and second set of attributes are selected from a list consisting of: facial attributes, positioning attributes and accessory attributes, wherein the facial attributes are selected from a list consisting of: an age, a gender, an ethnicity, an emotion, an expression, a complexion and an eye color, wherein the positioning attributes are selected from a list consisting of: a position of a face, an orientation of a face, a pose of a face, and an elevation of a face, and wherein the accessory attributes are selected from a list consisting of: existence of spectacles on a face, existence of jewelry on the face and existence of hair dressing accessories on the face.

16. The system of claim 12 , wherein the processor is configured to train the ML-based face generator module by:

receiving at least one labeled image of a face, pertaining to the labeled training set of images, as a first input;

receiving at least one output indication of an attribute classifier as a second input; and

receiving at least one output indication of a viewing entity as a third input.

17. The system of claim 12 , wherein the processor is configured to retrain the reduced dimension face generator model by using the already trained face decoder and training the reduced dimension face encoder using a cost function that decreases as a level of dissimilarity between the attributes of the face in the generated image and the attributes of the input image decreases and increases as the level of dissimilarity in identity between the face in the generated image and the face in the input image decreases.

18. The system of claim 12 , wherein the processor is configured to produce the anonymized face by:

using the face encoder, to produce, from the new face, a faceprint vector having a reduced dimension in relation to the new face; and

using the face decoder to generate the anonymized face from the faceprint vector, wherein the face decoder comprises a generative neural network (GNN) module.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 28, 2022
From: KUTA, ELIRAN; BLONDHEIM, SELLA; PERRY, GIL; HACOHEN, YOAV
To: DE-IDENTIFICATION LTD.
Reel/Frame 060920/0086 →
Continuity (3)
Continuation PCTIL2020050907 · Aug 19, 2020
Provisional Application 62888844 · Aug 19, 2019
Related Publication 20220172517A1 · Jun 2, 2022
Cited By (1)
US 12,505,648