Automatic correction of facial sentiment of portrait images
Techniques are disclosed for correcting facial sentiment of digital images. Facial data captured in a target image is analyzed to obtain facial-based sentiment. A favored sentiment is determined based at least in part on the facial-based sentiment. The favored sentiment is then applied to at least one face included in the target image that doesn't reflect the favored sentiment. Analyzing facial data may include detecting facial landmarks that are good indicators of sentiment (e.g., eyes, mouth, eyebrows, jawline, and nose). Such landmarks can be processed, with supervised machine learning, to detect the corresponding facial sentiment. A favored sentiment of the target image is thus identified, and can be applied to one or more non-compliant faces in the target image. In some embodiments, the favored sentiment can be further based on a plurality of additional sentiment indicators, including geo data, text, and/or other images associated with the target image.
1. A computer-implemented method for automatically correcting facial sentiment of a digital image, the method comprising:
receiving a request to correct facial sentiment in a given target image;
analyzing facial data of the target image to obtain facial-based sentiment;
determining a winning sentiment based on the facial-based sentiment; and
applying the winning sentiment to at least one face included in the target image, wherein applying the winning sentiment comprises
identifying faces in the target image that do not match the winning sentiment; and
modifying one or more landmark features of each face that does not match the winning sentiment, thereby producing an edited version of the target image.
2. The method of claim 1 , wherein analyzing facial data of the target image to obtain the facial-based sentiment comprises:
detecting a face in the target image;
detecting landmarks of the detected face;
processing, with supervised machine learning and classification, the landmarks of the detected face to obtain the facial-based sentiment; and
identifying and outputting the winning sentiment.
3. The method of claim 2 , wherein the supervised machine learning and classification used to process the landmarks of the detected face to obtain the facial-based sentiment is carried out using a support vector machine (SVM).
4. The method of claim 1 , further comprising at least one of:
analyzing other images related to the target image to obtain supplemental facial-based sentiment, wherein the other images are related to the target image based on a time and location at which they were captured;
analyzing geo data of the target image to obtain geo-based sentiment; and
analyzing textual data associated with the target image to obtain textual-based sentiment;
wherein the winning sentiment is further based on at least one of the textual-based sentiment, the supplemental facial-based sentiment, and the geo-based sentiment.
5. The method of claim 4 , wherein analyzing other images related to the target image to obtain the supplemental facial-based sentiment comprises:
identifying one or more additional images related to the target image;
detecting a face in the additional images;
detecting landmarks of the detected face;
processing, with supervised machine learning and classification, the landmarks of the detected face to obtain the supplemental facial-based sentiment;
tracking supplemental facial-based facial sentiment of a plurality of people captured in the additional images; and
identifying and outputting a most-favored facial sentiment of the additional images.
6. The method of claim 4 , wherein analyzing geo data of the target image to obtain the geo-based sentiment comprises:
detecting a geographic location at which the target image was captured; and
identifying and outputting the geo-based a-sentiment of the target image based on the geographic location.
7. The method of claim 4 , wherein analyzing textual data associated with the target image to obtain the textual-based sentiment comprises:
detecting textual content associated with the target image, wherein the textual content is at least one of within and proximate to the target image;
determining sentiment of the detected textual content; and
identifying and outputting the textual-based sentiment of the target image based on the determined.
8. A computer program product including one or more non-transitory machine readable mediums encoded with instructions that when executed by one or more processors cause a process to be carried out for automatically correcting facial sentiment of a digital image, the process comprising:
receiving a request to correct facial sentiment in a given target image;
analyzing facial data of the target image to obtain facial-based sentiment;
determining a winning sentiment based on the facial-based sentiment; and
applying the winning sentiment to at least one face included in the target image, wherein applying the winning sentiment comprises
identifying faces in the target image that do not match the winning sentiment; and
modifying one or more landmark features of each face that does not match the winning sentiment, thereby producing an edited version of the target image.
9. The computer program product of claim 8 , wherein analyzing facial data of the target image to obtain the facial-based sentiment comprises:
detecting a face in the target image;
detecting landmarks of the detected face;
processing, with supervised machine learning and classification, the landmarks of the detected face to obtain the facial-based sentiment; and
identifying and outputting the winning sentiment.
10. The computer program product of claim 9 , the process further comprising at least one of:
analyzing other images related to the target image to obtain supplemental facial-based sentiment;
analyzing geo data of the target image to obtain geo-based sentiment; and
analyzing textual data associated with the target image to obtain textual-based sentiment;
wherein the winning sentiment is further based on at least one of the textual-based sentiment, the supplemental facial-based sentiment, and the geo-based sentiment.
11. The computer program product of claim 10 , wherein the other images are related to the target image based on a time and location at which they were captured, and wherein analyzing the other images related to the target image to obtain the supplemental facial-based sentiment comprises:
identifying one or more additional images related to the target image;
detecting a face in the additional images;
detecting landmarks of the detected face;
processing, with supervised machine learning and classification, the landmarks of the detected face to obtain the supplemental facial-based sentiment;
tracking supplemental facial-based sentiment of a plurality of people captured in the additional images; and
identifying and outputting a most-favored facial sentiment of the additional images.
12. The computer program product of claim 10 , wherein analyzing geo data of the target image to obtain the geo-based sentiment comprises:
detecting a geographic location at which the target image was captured; and
identifying and outputting the geo-based a-sentiment of the target image based on the geographic location.
13. The computer program product of claim 10 , wherein analyzing textual data associated with the target image to obtain the textual-based sentiment comprises:
detecting textual content associated with the target image, wherein the textual content is at least one of within and proximate to the target image;
determining sentiment of the detected textual content; and
identifying and outputting the textual-based sentiment of the target image based on the determined sentiment.
14. A system for automatically correcting facial sentiment of a digital image, comprising:
an input/request circuit to receive a request to correct facial sentiment in a given target image;
a facial data analysis circuit to analyze facial data of the target image to obtain facial-based sentiment;
a voting circuit to determine a winning sentiment based on the facial-based sentiment; and
a face adjust circuit to apply the winning sentiment to at least one face included in the target image, wherein the face adjust circuit applies the winning sentiment to at least one face included in the target image by
identifying faces in the target image that do not match the winning sentiment, and
modifying one or more landmark features of each face that does not match the winning sentiment, thereby producing an edited version of the target image.
15. The system of claim 14 , wherein the facial data analysis circuit analyzes facial data of the target image to obtain facial-based sentiment by:
detecting each face in the target image;
detecting landmarks of each detected face;
processing, with supervised machine learning and classification, the landmarks for each face to detect the sentiment on the corresponding face, wherein the supervised machine learning and classification is carried out using a support vector machine (SVM); and
identifying and outputting a most-favored facial sentiment of the target image, the most-favored sentiment being the facial-based sentiment of target image.
16. The system of claim 14 , further comprising at least one of:
a related image analysis circuit to analyze other images related to the target image to obtain supplemental facial-based sentiment;
a geo data analysis circuit to analyze geo data of the target image to obtain geo-based sentiment; and
a textual analysis circuit to analyze textual data associated with the target image to obtain textual-based sentiment;
wherein the voting circuit determines the winning sentiment based on at least one of the text-based sentiment, the supplemental facial-based sentiment, and the geo-based sentiment.
17. The system of claim 16 , wherein the other images are related to the target image based on the time and location at which they were captured, and wherein the related image analysis circuit analyzes other images related to the target image to obtain supplemental facial-based sentiment by:
identifying one or more additional images related to the target image;
detecting each face in the additional images;
detecting landmarks of each face;
processing, with supervised machine learning and classification, the landmarks for each face to detect the sentiment on the corresponding face;
tracking facial sentiment of each person captured in the additional images; and
identifying and outputting a most-favored facial sentiment of the additional images, the most-favored sentiment being the supplemental facial-based sentiment of target image.
18. The system of claim 16 , wherein:
the geo data analysis circuit analyzes geo data of the target image to obtain geo-based sentiment by
detecting the geographic location at which the target image was captured, and
identifying and outputting a sentiment of the target image based on geographic location, the sentiment being the geo-based sentiment of target image; and
the textual analysis circuit analyzes textual data associated with the target image to obtain textual-based sentiment by
detecting textual content associated with the target image, wherein the textual content is at least one of within and proximate to the target image,
determining sentiment of the detected text, and
identifying and outputting a most-favored sentiment based on text of target image, the most-favored sentiment being the textual-based sentiment of target image.