Anonymization of facial expressions
A method facilitates training of an automatic facial expression recognition system through distributed anonymization of facial images, thereby allowing people to submit their own facial images without divulging their identities. Original facial images are accessed and perturbed to generate synthesized facial images. Personal identities contained in the original facial images are no longer discernable from the synthesized facial images. At the same time, each synthesized facial image preserves at least part of the emotional expression contained in the corresponding original facial image.
1. A computer-implemented method for generating synthesized facial images, the method comprising:
accessing an original facial image of a human subject; and
perturbing the original facial image to generate a synthesized facial image, the synthesized facial image no longer recognizable as the human subject but preserving at least part of the emotional expression of the original facial image;
wherein the synthesized facial image is no longer recognizable as the human subject when a probability that humans can correctly identify the human subject from the synthesized facial image is no greater than a threshold.
2. A computer-implemented method for generating synthesized facial images, the method comprising:
accessing an original facial image of a human subject; and
perturbing the original facial image to generate a synthesized facial image, the synthesized facial image no longer recognizable as the human subject but preserving at least part of the emotional expression of the original facial image;
wherein the step of perturbing the original facial image comprises:
encoding the original facial image as a feature set, the feature set containing personal identity components and expression components, the personal identity components contributing to recognizability of the facial image and the expression components contributing to the emotional expression of the facial image;
applying a perturbation transform to the feature set, wherein the perturbation transform substantially perturbs the personal identity components and substantially preserves at least some of the expression components; and
decoding the perturbed feature set to obtain the synthesized facial image.
3. The method of claim 2 wherein the step of encoding the original facial image as a feature set uses unsupervised learning.
4. The method of claim 2 wherein the step of encoding the original facial image as a feature set uses principal component analysis.
5. The method of claim 2 wherein the step of encoding the original facial image as a feature set uses independent component analysis.
6. The method of claim 2 wherein the step of encoding the original facial image as a feature set uses sparse coding.
7. The method of claim 2 wherein the step of encoding the original facial image as a feature set uses filter banks.
8. The method of claim 2 wherein the step of encoding the original facial image as a feature set uses Gabor bandpass filters.
9. The method of claim 2 wherein the step of applying a perturbation transform to the feature set comprises setting some values in the feature set to zero.
10. The method of claim 2 wherein the step of applying a perturbation transform to the feature set comprises adding noise to the feature set.
11. The method of claim 2 wherein the step of applying a perturbation transform to the feature set comprises permuting some values in the feature set.
12. The method of claim 2 wherein the step of applying a perturbation transform to the feature set comprises applying a linear transform to some values in the feature set.
13. The method of claim 2 wherein the step of applying a perturbation transform to the feature set comprises applying a linear discriminant analysis to the feature set.
14. The method of claim 2 wherein the step of decoding the perturbed feature set uses supervised learning.
15. The method of claim 14 wherein the supervised learning uses a training set that includes unrecognizable facial images and their corresponding original facial images.
16. The method of claim 2 wherein the step of decoding the perturbed feature set uses SVM regression.
17. The method of claim 2 wherein the step of decoding the perturbed feature set uses multilayer perceptron.
18. The method of claim 2 further comprising:
confirming that the synthesized facial image is no longer recognizable as the human subject.
19. The method of claim 18 wherein the confirming step uses a recognition algorithm to confirm that the synthesized facial image is no longer recognizable as the human subject.
20. The method of claim 18 wherein the confirming step uses discriminant analysis to confirm that the synthesized facial image is no longer recognizable as the human subject.
21. The method of claim 18 wherein the confirming step uses human crowdsourcing to confirm that the synthesized facial image is no longer recognizable as the human subject.
22. The method of claim 2 further comprising:
receiving an indication from the human subject whether the synthesized facial image is acceptable to the human subject as no longer recognizable as the human subject.
23. A computer-implemented method for generating synthesized facial images, the method comprising:
accessing an original facial image of a human subject; and
perturbing the original facial image to generate a synthesized facial image, the synthesized facial image no longer recognizable as the human subject but preserving at least part of the emotional expression of the original facial image,
wherein expression elements in the set of original facial images are in aggregate also present in the set of synthesized facial images and the step of perturbing the original facial images comprises:
perturbing an original facial image in at least M ways to generate a group of M synthesized facial images from the original facial image, M>1, wherein each synthesized facial image contains less than all of the expression elements in the original facial image but the group of synthesized facial images in the aggregate contains substantially all of the expression elements in the original facial image.
24. The method of claim 23 wherein different synthesized facial images from the group are based on different spatial regions from the original facial image.
25. The method of claim 23 wherein different synthesized facial images from the group are based on different facial features from the original facial image.
26. The method of claim 23 wherein different synthesized facial images from the group are based on different spatial frequency bands from the original facial image.
27. A non-transitory tangible computer readable medium containing instructions that, when executed by a processor, execute a method for generating synthesized facial images, the method comprising:
accessing an original facial image of a human subject; and
perturbing the original facial image to generate a synthesized facial image, the synthesized facial image no longer recognizable as the human subject but preserving at least part of the emotional expression of the original facial image,
wherein the step of perturbing the original facial image comprises:
encoding the original facial image as a feature set, the feature set containing personal identity components and expression components, the personal identity components contributing to recognizability of the facial image and the expression components contributing to the emotional expression of the facial image;
applying a perturbation transform to the feature set, wherein the perturbation transform substantially perturbs the personal identity components and substantially preserves at least some of the expression components; and
decoding the perturbed feature set to obtain the synthesized facial image.
28. The computer readable medium of claim 27 wherein the step of encoding the original facial image as a feature set uses unsupervised learning.
29. The computer readable medium of claim 27 wherein the step of encoding the original facial image as a feature set uses principal component analysis.
30. The computer readable medium of claim 27 wherein the step of encoding the original facial image as a feature set uses independent component analysis.
31. The computer readable medium of claim 27 wherein the step of encoding the original facial image as a feature set uses sparse coding.
32. The computer readable medium of claim 27 wherein the step of encoding the original facial image as a feature set uses filter banks.
33. The computer readable medium of claim 27 wherein the step of encoding the original facial image as a feature set uses Gabor bandpass filters.
34. The computer readable medium of claim 27 wherein the step of applying a perturbation transform to the feature set comprises setting some values in the feature set to zero.
35. The computer readable medium of claim 27 wherein the step of applying a perturbation transform to the feature set comprises adding noise to the feature set.
36. The computer readable medium of claim 27 wherein the step of applying a perturbation transform to the feature set comprises permuting some values in the feature set.
37. The computer readable medium of claim 27 wherein the step of applying a perturbation transform to the feature set comprises applying a linear transform to some values in the feature set.
38. The computer readable medium of claim 27 wherein the step of applying a perturbation transform to the feature set comprises applying a linear discriminant analysis to the feature set.
39. The computer readable medium of claim 27 wherein the step of decoding the perturbed feature set uses supervised learning.
40. The computer readable medium of claim 39 wherein the supervised learning uses a training set that includes unrecognizable facial images and their corresponding original facial images.
41. The computer readable medium of claim 27 wherein the step of decoding the perturbed feature set uses SVM regression.
42. The computer readable medium of claim 27 wherein the step of decoding the perturbed feature set uses multilayer perceptron.
43. The computer readable medium of claim 27 further comprising:
confirming that the synthesized facial image is no longer recognizable as the human subject.
44. The computer readable medium of claim 43 wherein the confirming step uses a recognition algorithm to confirm that the synthesized facial image is no longer recognizable as the human subject.
45. The computer readable medium of claim 43 wherein the confirming step uses discriminant analysis to confirm that the synthesized facial image is no longer recognizable as the human subject.
46. The computer readable medium of claim 43 wherein the confirming step uses human crowdsourcing to confirm that the synthesized facial image is no longer recognizable as the human subject.
47. The computer readable medium of claim 27 further comprising:
receiving an indication from the human subject whether the synthesized facial image is acceptable to the human subject as no longer recognizable as the human subject.
48. A system for generating synthesized facial images, the system comprising:
an encoder that encodes the original facial image as a feature set, the feature set containing personal identity components and expression components, the personal identity components contributing to recognizability of the facial image and the expression components contributing to the emotional expression of the facial image;
a perturbation engine coupled to the encoder, that applies a perturbation transform to the feature set, wherein the perturbation transform substantially perturbs the personal identity components and substantially preserves at least some of the expression components; and
a decoder coupled to the perturbation engine, that decodes the perturbed feature set to obtain the synthesized facial image.
49. A non-transitory tangible computer readable medium containing instructions that, when executed by a processor, execute a method for generating synthesized facial images, the method comprising:
accessing an original facial image of a human subject; and
perturbing the original facial image to generate a synthesized facial image, the synthesized facial image no longer recognizable as the human subject but preserving at least part of the emotional expression of the original facial image;
wherein expression elements in the set of original facial images are in aggregate also present in the set of synthesized facial images and the step of perturbing the original facial images comprises:
perturbing an original facial image in at least M ways to generate a group of M synthesized facial images from the original facial image, M>1, wherein each synthesized facial image contains less than all of the expression elements in the original facial image but the group of synthesized facial images in the aggregate contains substantially all of the expression elements in the original facial image.
50. The computer readable medium of claim 49 wherein different synthesized facial images from the group are based on different spatial regions from the original facial image.
51. The computer readable medium of claim 49 wherein different synthesized facial images from the group are based on different facial features from the original facial image.
52. The computer readable medium of claim 49 wherein different synthesized facial images from the group are based on different spatial frequency bands from the original facial image.