Emoji manipulation using machine learning
A machine learning system is accessed. The machine learning system is used to translate content into a representative icon. The machine learning system is used to manipulate emoji. The machine learning system is used to process an image of an individual. The machine learning processing includes identifying a face of the individual. The machine learning processing includes classifying the face to determine facial content using a plurality of image classifiers. The classifying includes generating confidence values for a plurality of action units for the face. The facial content is translated into a representative icon. The translating the facial content includes summing the confidence values for the plurality of action units. The representative icon comprises an emoji. A set of emoji can be imported. The representative icon is selected from the set of emoji. The emoji selection is based on emotion content analysis of the face.
1. A computer-implemented method for emoji manipulation comprising:
accessing a machine learning system comprising a convolutional neural network wherein the convolutional neural network includes an input layer, one or more hidden layers, and a classification layer;
processing an image of an individual, using the machine learning system, wherein the processing includes:
identifying a face of the individual; and
classifying the face to determine facial content based on a facial expression from the face, wherein the classifying includes identifying the facial expression using a hidden layer from the one or more hidden layers, using a plurality of image classifiers, wherein the classifying includes generating confidence values for a plurality of action units for the face; and
translating the facial content into a representative icon, wherein the translating the facial content includes:
summing the confidence values for the plurality of action units, wherein the plurality of action units includes at least one positive action unit and at least one negative action unit; and
computing a weight corresponding to the representative icon, wherein the weight is computed by a minimum function, and wherein the minimum function includes as input the at least one positive action unit and the at least one negative action unit.
2. The method of claim 1 wherein the representative icon comprises an emoji.
3. The method of claim 1 wherein the summing includes a weighted summation of the confidence values.
4. The method of claim 3 wherein the summing includes negative weights.
5. The method of claim 1 further comprising performing alignment on the face that was identified.
6. The method of claim 5 further comprising performing normalization on the face that was identified.
7. The method of claim 1 wherein the identifying further comprises identifying a second face within the image.
8. The method of claim 7 further comprising providing a second representative icon for the second face.
9. The method of claim 1 wherein the translating is further based on motion of regions within the face of the individual.
10. The method of claim 1 further comprising tracking the face in the image and a plurality of additional images.
11. The method of claim 10 further comprising tracking a second face within the plurality of additional images.
12. The method of claim 11 further comprising selecting the individual from a plurality of people.
13. The method of claim 10 further comprising selecting a most emoted image from the image and the plurality of additional images and performing the translating for the most emoted image.
14. The method of claim 1 wherein the representative icon comprises a static image emoji.
15. The method of claim 1 wherein the representative icon comprises an animated emoji image.
16. The method of claim 1 wherein the representative icon comprises a cartoon emoji representation.
17. The method of claim 1 wherein the representative icon comprises an emoji containing information on gender, age, or ethnicity.
18. The method of claim 17 wherein the gender, age, or ethnicity is detected by analyzing the image.
19. The method of claim 1 further comprising providing a plurality of emoji wherein the representative icon is included within the plurality of emoji.
20. The method of claim 19 wherein the plurality of emoji is presented to the individual for selection of an appropriate emoji.
21. The method of claim 20 wherein the plurality of emoji is presented with a probability score to aid the individual in the selection.
22. The method of claim 20 wherein a subset of the plurality of emoji is highlighted based on the facial expression.
23. The method of claim 1 further comprising calculating a facial expression metric for the individual based on the classifying.
24. The method of claim 23 further comprising generating a cognitive state metric that is a function of engagement level for the individual based on the facial expression metric.
25. The method of claim 24 further comprising including the cognitive state metric in the translating.
26. The method of claim 1 further comprising training the convolutional neural network wherein the training includes assigning weights to inputs for one or more layers within the convolutional neural network.
27. The method of claim 1 wherein the convolutional neural network further includes a bottleneck layer to accomplish determination of facial content.
28. The method of claim 1 wherein the convolutional neural network further comprises a convolutional layer and a fully connected layer.
29. The method of claim 28 wherein the fully connected layer is configured to process a plurality of data points from the convolutional layer.
30. The method of claim 1 wherein the at least one negative action unit indicates absence of the action unit.
31. A computer system for image analysis comprising:
a memory which stores instructions;
one or more processors coupled to the memory wherein the one or more processors, when executing the instructions which are stored, are configured to:
access a machine learning system comprising a convolutional neural network wherein the convolutional neural network includes an input layer, one or more hidden layers, and a classification layer;
process an image of an individual, using the machine learning system, wherein the processing includes:
identifying a face of the individual; and
classifying the face to determine facial content based on a facial expression from the face, wherein the classifying includes identifying the facial expression using a hidden layer from the one or more hidden layers, using a plurality of image classifiers, wherein the classifying includes generating confidence values for a plurality of action units for the face; and
translate the facial content into a representative icon, wherein the translating the facial content includes:
summing the confidence values for the plurality of action units, wherein the plurality of action units includes at least one positive action unit and at least one negative action unit; and
computing a weight corresponding to the representative icon, wherein the weight is computed by a minimum function, and wherein the minimum function includes as input the at least one positive action unit and the at least one negative action unit.
32. A computer program product embodied in a non-transitory computer readable medium for image analysis, the computer program product comprising code which causes one or more processors to perform operations of:
accessing a machine learning system comprising a convolutional neural network wherein the convolutional neural network includes an input layer, one or more hidden layers, and a classification layer;
processing an image of an individual, using the machine learning system, wherein the processing includes:
identifying a face of the individual; and
classifying the face to determine facial content based on a facial expression from the face, wherein the classifying includes identifying the facial expression using a hidden layer from the one or more hidden layers, using a plurality of image classifiers, wherein the classifying includes generating confidence values for a plurality of action units for the face; and
translating the facial content into a representative icon, wherein the translating the facial content includes:
summing the confidence values for the plurality of action units, wherein the plurality of action units includes at least one positive action unit and at least one negative action unit; and
computing a weight corresponding to the representative icon, wherein the weight is computed by a minimum function, and wherein the minimum function includes as input the at least one positive action unit and the at least one negative action unit.